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Preface 


The  purpose  of  this  study  was  to  evaluate  the  potential 
of  two  measures,  percent  defects  and  cycle  time,  to  measure 
overall  performance  of  the  Air  Force  Critical  Item  Program 
(CIP).  The  need  for  such  study  was  generated  from  the 
current  absence  of  any  measurement  system  available  to 
evaluate  the  CIP.  This  research  used  these  two  measures 
with  actual  data,  evaluated  the  correlation  between  aircraft 
availability  and  percent  defects  and  cycle  time,  and 
analyzed  management  perceptions  addressing  the  usefulness  of 
these  measures  for  field  use.  Performance  of  this  analysis 
resulted  in  support  for  these  measures  by  over  two  thirds  of 
the  critical  item  managers  interviewed.  As  a  result,  this  ■ 
study  concluded  that  both  of  these  measures,  percent  defects 
and  cycle  time,  could  be  useful  to  the  Air  Force  in  managing 
the  CIP  effectively. 

Several  individuals  were  instrumental  in  the  successful 
completion  of  this  study.  First  and  foremost  in  this  effort 
were  the  contributions  of  my  thesis  advisor  and  reader, 

Lt  Col  Larry  W.  Emmelhainz  and  Capt  John  Sullivan.  These 
two  individuals  were  of  invaluable  assistance  in  guiding  the 
proper  direction  of  this  research.  I  am  also  deeply 
indebted  to  all  others  who  provided  insight  and  direction  of 
the  problem  at  hand.  In  this  respect,  I  would  especially 
like  to  thank  the  assistance  of  Mr.  Luis  Correa,  Mr.  Ed 

i  i 


Kroutsdor f er ,  and  Mr.  Cristopher  Lynch.  I  would  also  like 
to  thank  the  efforts  of  Capt  Jill  Page  and  Ms.  Helen 
Hartness  who  were  instrumental  in  providing  actual  data, 
without  which  this  study  would  not  have  been  successful. 
Finally,  I  extend  my  deep  gratitude  to  all  critical  item 
managers  who  were  involved  in  the  interview  process.  Their 
input  and  opinions  were  the  cornerstone  upon  which  the 
conclusions  of  this  study  lay. 
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AN  ANALYSIS  OF  MEASURES  USED  TO  EVALUATE 


THE  AIR  FORCE  CRITICAL  ITEM  PROGRAM 


I  .  Introduction 


General  Issue 

The  Air  Force  critical  item  program  is  a  dynamic 
management  process  crucial  to  the  effective  and  efficient 
support  of  Air  Force  weapon  systems.  It  is  a  program  that 
has  been  gradually  developing  from  1967  to  present.  During 
this  time  the  program  has  suffered  difficulties  preventing 
it  from  achieving  its  full  potential.  One  of  the  current 
shortfalls  of  the  program  is  the  nonexistence  of  measures  to 
evaluate  its  performance.  Recognizing  this  shortfall,  this 
study  evaluates  two  quality  control  measures  for  use  in 
evaluating  the  performance  of  the  Air  Force  Critical  Item 
Program. 

Background 

Even  more  than  before,  the  progress  of  any  war  that 
involves  something  more  than  a  walkover  will  be  in  part 
determined  by  the  speed  with  which  material  can  be 
moved  from  the  rear  to  the  front  and  allocated  to.  the 
units  most  in  need.  (20:69) 

The  rapid  build-up  of  American  forces  during  Operation 
Desert  Shield  resulted  in  the  re-identification  of 
shortfalls  associated  with  current  Air  Force  Information 
Systems.  The  C-141  System  Program  Manager  identified  these 
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shortfalls  during  surge  of  Air  Logistic  Center  (ALC) 
activities  while  supporting  Operation  Desert  Shield  (21) 

The  basic  cause  identified  was  the  inability  of  Air 
Force  Information  Systems  to  provide  accurate  asset 
reporting  information  vital  to  the  management  of  surge 
efforts  (21).  Without  such  information,  the  System  Program 
Manager's  ability  to  maJte  crucial  surge  management  decisions 
markedly  decreases.  The  existing  systems  generated 
information  that,  according  to  SAFE  reports,  dated  back  as 
much  as  a  year.  Other  reports  produced  data  which  required 
extensive  manual  manipulation  to  provide  up-to-date  asset 
levels  and  specific  support  problem  areas  (21).  Based  on 
these  shortfalls,  the  C-141  System  Program  Manager  (SPM) 
requested  a  study  be  conducted  to  determine  what  information 
systems,  if  any,  were  capable  of  supporting  surge  management 
requirements . 

Further  investigation  of  the  surge  process  and 
available  supporting  information  systems  uncovered  several 
Air  Force  management  and  development  efforts  in  this 
problem  area.  These  discoveries  helped  focus  this  study 
specifically  on  problems  surrounding  the  heart  of  the  surge 
operation  --  the  critical  item  selection  process. 

Critical  Item  Management 

As  stated  earlier,  the  C-141  SPM  identified  a  problem 
in  extracting  accurate  information  to  make  time-sensitive 
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surge  decisions.  To  understand  the  impact  of  this  shortfall 
a  review  of  the  surge  management  process  is  beneficial. 

At  the  start  of  the  surge  process,  the  SPM  must 
establish  a  prioritized  critical  item  list  prior  to  fully 
coordinating  and  implementing  other  activities.  Foremost  in 
this  process  is  the  identification  of  those  items  in 
critical  support  status  most  contributing  to  a  weapons 
system's  inability  to  perform  Its  primary  mission  (21).  A 
critical  item  is  determined  by  applying  certain  criteria 
measuring  the  unavailability  of  that  item  on  its  respective 
weapon  system.  A  critical  item  is  officially  defined  by  AFM 
67-1,  Volume  1,  Part  One,  Chapter  26  as  "those  items  having 
the  most  detrimental  impact  on  mission  performance" 
(15:26-9). 

Upon  attainment  of  a  prioritized  critical  item  list, 
activities  such  as  realignment  of  personnel,  expansion  of 
the  work  week,  expedition  of  contracts,  and  expedition  of 
repair  can  be  focussed  to  those  items  with  highest  priority 
on  the  critical  item  list  (29:4).  Items  crucial  to  mission 
performance,  such  as  aircraft  landing  gear,  qualify  as 
critical  items  if  they  meet  specific  major  command  (MAJCOM) 
criteria  for  degraded  support.  Items  identified  as 
critical  receive  increased  management  and  resource  support 
with  the  goal  of  maximizing  weapon  system  availability. 
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History  of  Critical  Item  Management 

In  some  form,  the  Air  Force  Critical  Item  Program  has 
existed  since  1967.  The  first  critical  item  program  was 
called  the  "High  NORS  (Not  Operationally  Ready  Supply) 
Critical  Item  Program”  (43:1).  Like  the  current  program, 
the  goal  of  the  High  NORS  Critical  Item  Program  was  to 
increase  weapon  system  availability  by  improving  critical 
item  asset  support  (43:1). 

Through  the  years  several  studies  and  audits  have 
identified  flaws  in  this  program.  Starting  in  1967,  these 
studies  addressed  subjects  such  as  problem  forecasting, 
development  of  mathematical  formulae,  criteria  bias  and 
evaluation,  appropriate  management  action,  and  model 
development  (40;  48;  11;  43;  2;  4).  A  1986  study  by  the  Air 
Force  Logistics  Management  Center  (AFLMC),  served  to 
redefine  the  critical  item  management  process  (7),  This 
study,  conducted  through  MAJCOM  and  AFLC  request,  analyzed 
the  criteria  and  procedures  used  to  identify  critical  items 
(7:iii).  Conclusions  from  this  study  recognized  that  the 
CIP  was  reactive,  did  not  consider  sustainability  shortages, 
was  one  dimensional  (considered  only  one  criterion),  was 
manually  work  load  intensive,  lacked  communication  between 
item  managers  and  the  MAJCOMs,  and  had  corrective  actions 
which  indirectly  reduced  supportabi 1 ity  by  reducing 
worldwide  demand  rates  (7:29).  As  a  result  of  this  study, 
the  Air  Force  implemented  a  regulation  attempting  to 
formalize  the  AFLMC  recommendation  to  require  the  use  of 
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multidimensional  criteria  (7:29).  The  structur^:  of  the 
current  CIP  is  based  on  the  conclusions  and  supporting 
recommendations  of  this  report. 

Current  Status  of  Critical  Item  Management 

Application  of  the  AFLMC  recommendations  and  the 
introduction  of  improved  management  information  system 
capabilities  have  provided  today's  managers  with  potentially 
powerful  management  tools.  In  efforts  to  improve  the 
program,  AFLMC  recommendations  brought  about  the  institution 
of  a  three  tiered  categor i2ation  for  degraded  support  items. 
These  three  tiers  cited  from  least  to  most  important  are 
"Problem,"  "Potential  Critical,"  and  "Critical,"  (15:26-9). 
Under  the  new  guidance,  items  subject  to  critical  item 
management  should  meet  the  third  hurdle  (critical),  while 
items  in  the  first  two  hurdles  were  to  be  watched  carefully 
(7:20-23)  . 

Current  guidance  requires  that  CIP  selections  be  based 
on  one  or  more  of  these  criteria  (15:26-8).  These  criteria 
(discussed  in  detail  in  Chapter  II)  have  nine  categories 
including  weapon  system  factor,  monthly  mission  capable 
(MICAP)  hours,  monthly  MICAP  incidents,  yearly  MICAP 
incidents,  yearly  awaiting  parts  (AWP)  incidents,  single 
point  failures,  MAJCOM  developed  parameters,  and  supply 
assistance  follow-up  procedures  (15:26-8).  Individual 
MAJCOMs  are  responsible  for  tailoring  specific  guidelines 
within  these  criteria  to  establish  critical  item  selection 
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(15:26-5).  Based  on  current  guidance,  MAJCOMs  have  final 
approval  authority  for  all  items  included  and  excluded  from 
the  CIP  (15:26-10  )  . 

Relatively  new  management  information  systems  are  also 
available  to  aid  managers  in  the  critical  item  selection 
process.  The  Weapon  System  Management  Information  System 
(WSMIS)/Get-Well  Assessment  Module  (GWAM)  (22)  is 
specifically  designed  for  this  purpose.  WSMIS/GWAM,  based 
on  input  from  numerous  other  Air  Force  information  systems, 
generates  a  critical  item  candidate  list  (22:2-4).  This 
list  is  individually  tailored  to  each  MAJCOM  by  using 
specific  criteria  to  analyze  items  recommended  to  the 
critical  item  candidate  list.  In  turn,  each  MAJCOM  and  SPM 
then  accomplish  a  review  of  the  critical  item  candidate  list 
to  identify  those  few  exceptions  not  identified  by 
WSMIS/GWAM  that  they  judge  should  be  elevated  to  critical 
item  status.  Ideally,  the  WSMIS/GWAM  list  should  identify 
60-80  percent  of  the  those  items  approved  as  critical  by  the 
MAJCOMs  (36).  A  list  with  this  accuracy  would  enable  the 
MAJCOMs  to  rely  on  WSMIS/GWAM  for  primary  critical  item 
selection  rather  than  the  manpower  intensive  manual  method 
currently  used.  Because  a  cap  exists  on  the  maximum  number 
of  items  in  the  CIP,  the  MAJCOM  and  SPM  are  required  to 
negotiate  the  contents  of  the  final  list  (15:26-7).  Items 
have  to  be  eliminated  if  more  critical  items  are  identified 
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than  officially  authorized.  As  stated  earlier,  the  MAJCOM 
has  final  authority  on  all  items  included  or  excluded  in  the 
CIP. 

Problems  Associated  With  the  Current  Critical  Item  Program 

In  spite  of  the  changes  made  in  the  late  1980 *s,  a 
recent  audit  by  the  Air  Force  Audit  Agency,  AFAA/QLS,  has 
identified  numerous  problems  with  the  CIP,  Audit  Agency 
findings  include  inconsistencies  in  CIP  guidance,  low 
utilization  of  WSMIS/GWAM  management  products,  inconsistent 
use  of  MAJCOM  criteria,  and  misuse  of  the  CIP  for  less  than 
critical  items  (12). 

The  Audit  maintains  that  CIP  official  guidance 
contradicts  itself  (12).  According  to  AFM  67-1,  Volume  I, 
Part  One,  Chapter  26,  Section  7,  Paragraph  a,  critical  item 
selection  will  be  based  on  multidimensional  criteria. 

Shortly  following  in  this  same  paragraph,  it  states  a 
requirement  that  only  one  criterion  is  necessary  to  justify 
critical  item  selection  (15:26-8).  In  addition  to  being 
confusing,  this  guidance  defeats  the  purpose  of 
multidimensional  criteria  by  allowing  selection  of  critical 
items  based  on  only  one  criterion  (12).  The  guidance  also 
contradicts  conclusions  stated  in  the  AFLMC  report  which 
identified  the  inadequacy  of  one  dimensional  criterion  for 
critical  item  selection  (7:29). 

The  Audit  Agency  conducted  random  sampling  of 
information  systems  associated  with  the  critical  item 
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management  process.  These  samples  were  taken  from  Strategic 
Air  Command  (SAC)  and  Tactical  Air  Command  (TAC)  programs 
from  14  June  1990  to  1  August  1990  (12).  Results  of  the 
sampling  are  as  follows: 

SAC/14  June  1990.  Of  206  items  on  the  WSMIS/GWAM 
Critical  Item  Candidate  list  for  SAC,  30  items  were 
randomly  selected.  Of  these  30  items,  16  items  (53%) 
were  not  identified  in  the  MAJCOM  approved  CIP  list. 

SAC/1  July  1990.  Of  202  items  in  the  SAC  CIP,  30 
items  were  randomly  selected.  Of  these  30  items,  16 
items  (53%)  had  not  been  identified  as  critical  by 
WSMIS/GWAM. 

TAC/1  August  1990.  Of  318  items  in  the  TAC  CIP,  30 
items  were  randomly  selected.  Of  these  30  items,  11 
items  (36%)  were  not  on  the  WSMIS/GWAM  Critical  Item 
Candidate  list. 

TAC/1  August  1990.  Of  403  items  coded  critical  on  the 
WSMIS/GWAM  Critical  Item  Candidate  list  for  TAC,  30 
items  were  randomly  selected.  Of  these  30  items,  16 
items  (53%)  were  not  identified  in  the  MAJCOM  approved, 
CIP.  (12) 

These  samples  strongly  indicate  that  MAJCOM  selected 
critical  items  and  those  selected  by  WSMIS/GWAM  are 
inconsistent . 

Other  Air  Force  Audit  Agency  reviews  identified 
numerous  items  in  the  CIP  which  did  not  meet  any  of  the 
MAJCOM  specified  criteria  (12).  Many  of  these  items  had 
been  in  the  program  for  several  months  (up  to  nine  months  in 
some  instances)  while  never  meeting  critical  status 
criteria,  including  the  time  of  their  initial  entry  to  the 
CIP  (12).  Current  guidance  (15:26-12)  requi 'es  deletion  of 
CIP  items  from  the  program  if  they  meet  standard  thresholds 
for  the  previous  three  months.  Items  attaining  these 
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thresholds  are  referred  to  as  "well."  Audit  findings  that 
"well"  items  remain  in  the  CIP  in  upwards  of  nine  months 
suggests  a  lack  of  proper  management  attention  to  the 
program.  It  is  appropriate  to  reiterate  that  for  every  item 
in  the  CIP,  another  critical  item  not  in  the  CIP  is 
prevented  entry  due  to  the  established  cap.  Thus  when 
"well"  items  remain  unchecked,  limited  funding  and  resources 
are  needlessly  wasted.  This  finding  suggests  that 
management  must  be  more  attentive  to  ensure  items  remain  in 
the  CIP  only  as  long  as  necessary. 

Telephone  interviews  with  critical  item  managers  in  the 
field  indicated  a  low  confidence  level  in  items  identified 
by  the  WSMIS/GWAM  critical  item  candidate  list  (52;  54). 
These  managers  indicated  that  because  of  the  low  correlation 
of  the  WSMIS/GWAM  critical  item  candidate  list  to  the  MAJCOM 
approved  critical  item  list,  the  WSMIS/GWAM  list  was 
currently  of  little  value.  This  last  observation  is  very 
disturbing  to  managers  due  to  the  enormous  sum  of  money 
invested  in  WSMIS/GWAM  to  automate  the  critical  item 
selection  process  (36). 

In  summary,  the  critical  item  management  process  has 
been  in  existence  for  over  two  and  a  half  decades.  It  was 
originally  designed  to  be  a  proactive  management  tool  with 
the  goal  of  improving  weapon  system  availability.  Over  the 
past  two  and  half  decades,  numerous  Air  Force  studies  and 
audits  have  identified  significant  and  recurring  defects  in 
the  program.  Implementation  of  study  and  audit 
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recommendations  have  been  very  slow.  As  a  result,  these 
defects  continue  to  limit  the  achievement  of  the  program's 
true  potential.  Though  management  has  addressed 
deficiencies  on  an  individual  basis,  there  are  currently  no 
measures  to  evaluate  the  performance  of  the  program  as  a 
whole  (36).  Thus,  inherent  defects  within  the  program  are 
retained  and  continue  to  impair  its  performance.  To  address 
this  situation,  the  quality  literature  suggests  measures 
could  be  instituted  which  might  provide  management  with  an 
overall  perspective  of  the  CIPs  performance.  With 
measurement/control  devices  in  place,  the  Air  Force  could 
incrementally  identify  and  correct  individual  problems 
affecting  the  success  of  the  CIP. 

Quality  Control  Applications 

The  ideal  unit  of  measure  provides  an  agreed  basis  for 
decision  making... is  understandable. . .applies 
broadly... is  susceptible  of  uniform  interpretat ion . . . is 
economic  to  apply... and  is  compatible  with  existing 
designs  of  sensors.  (32:76-78) 

Review  of  quality  control  literature  indicates  that  the 
use  of  statistical  quality  control  (SQC)  techniques  are 
applicable  to  the  problems  encountered  by  the  CIP  (8;  10; 

23;  24;  32;  39;  55;  50).  Specifically  of  interest  are  two 
measures,  percent  defects  and  cycle  time,  which  have  been 
used  by  several  top  quality  management  performers.  These 
measures  have  aided  corporations  in  attaining  substantial 
quality  improvements  that  would  not  have  otherwise  been 
achieved  (50;  37:108). 
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The  first  measure  of  interest  is  percent  defects. 

Unlike  the  common  meaning  of  defective,  in  the  context  of 
SQC  it  means  a  failure  to  "conform  to  specifications  in  some 
respect"  (24:17-18).  Therefore,  a  defect  is  a  single 
failure  to  conform  to  one  specification  (24:18).  Using 
these  definitions  of  defective  and  defect,  the  measure  of 
percent  defects  can  be  applied  to  a  service  or  production 
application.  According  to  Juran,  percent  defects  falls 
under  his  third  species  for  units  of  measure  associated  with 
errors  and  failures  (32:72).  Looking  specifically  at  the 
CIP,  several  conditions  can  be  defined  as  a  defect.  Even 
though  each  MAJCOM  establishes  tailored  critical  item 
selection  criteria,  standard  defect  conditions  can  be 
globally  applied  to  the  program  as  a  whole.  Examples  of 
these  defect  conditions  include: 

1.  Items  in  the  CIP  which  do  not  meet  established 
critical  item  selection  criteria.  For  example,  items 
achieving  a  "well"  status  that  continue  to  be  carried  in  the 
CIP  . 

2.  Items  that  do  meet  critical  item  selection  criteria 
that  are  not  in  the  CIP. 

3.  Items  that  enter  and  leave  the  CIP  on  a  recurring 
bas is  . 

Using  percent  defects  as  a  measure,  the  Air  Force  could 
establish  a  tool  that  identifies  changes  in  CIP  performance. 
This  should  encourage  CIP  managers  to  decrease  the  number  of 
defects  and  create  an  environment  for  continual  improvement. 
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As  indicated  earlier,  recent  application  of  this  quality 
control  measure  has  resulted  in  substantial  quality 
improvements  for  private  industrial  firms  such  as  Motorola 
and  Xerox  (37:104,108;  50).  The  improvements  made  by  these 
two  firms  clearly  demonstrate  the  considerable  benefits  of 
using  SQC  methods » 

The  measure  of  percent  defects  by  itself,  however,  is 
not  enough.  A  measure  is  needed  that  addresses  the  problem 
of  items  remaining  in  the  CIP  longer  than  necessary.  This 
type  of  measurement  device  must  be  established  to  provide 
incentive  for  managers  to  improve  the  status  of  critical 
items  and  to  ensure  the  overall  program  continues  to  improve 
(17).  Without  such  a  measure,  managers  might  be  tempted  to 
manipulate  the  program  to  give  the  perception  of  better 
performance  when  in  reality  there  is  no  improvement.  Cycle 
time  provides  such  a  measure.  This  measure  falls  under 
Juran's  first  classification  of  measure  at  the  technological 
level  (32:72).  In  the  context  of  this  study,  cycle  time  is 
defined  as  the  time  spanning  from  when  an  item  enters  the 
CIP  to  the  time  an  item  leaves.  Like  percent  defects,  cycle 
time  is  also  used  by  Motorola  in  its  SQC  program.  Benefits 
derived  from  this  measure  have  decreased  delivery  rates  for 
Motorola  by  over  27%  (37:108).  Cycle  time  also  appears 
potentially  applicable  to  all  MAJCOMs. 
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While  the  CIP  is  known  not  to  work  well,  no  specific 


measures  of  the  program  exist.  The  quality  literature 
suggests  that  two  measures,  percent  defects  and  cycle  time, 
might  work;  however,  no  one  has  evaluated  these  measures 
with  respect  to  managing  the  CIP. 

Overall  Objective 

Together,  measurement  of  percent  defects  and  cycle  time 
could  identify  trends  in  the  performance  of  the  CIP  with 
respect  to  meeting  its  goal  of  increasing  aircraft 
availability.  Identification  of  these  trends  would  enable 
the  Air  Force  to  react  proactively  in  resolving  problems  and 
institute  continuous  improvement.  Therefore,  the  overall 
objective  of  this  thesis  is  to  test  the  proposition  that  the 
measures  of  percent  defects  and  cycle  time  can  provide  the 
Air  Force  with  a  useful  tool  for  evaluating  the  performance 
of  the  CIP. 

Investigative  and  Measurement  Questions.  To  answer  the 
question  posed  in  the  problem  statement,  two  investigative 
questions  are  addressed.  In  turn,  to  satisfy  each 
investigative  question,  several  measurement  questions  are 
identified.  Methodology  for  the  determination  of  problem 
statement  conclusions  are  discussed  in  detail  in  Chapter 
III.  The  two  investigative  questions  to  be  applied  and 
their  associated  measurement  questions  are  as  follows: 


Investigative  Question  #1.  Does  measurement  of 
percent  defects  aid  the  Air  Force  in  managing  the  CIP 
effectively? 

Associated  Measurement  Questions; 

1.  Do  items  in  the  CIP  meet  MAJCOM  criteria 

lAW  AFM  67-1? 


2.  Do  items  in  the  CIP  correspond  to  items 
identified  by  WSMIS/GWAM? 

3.  Do  items  enter  and  leave  the  program  on  a 
recurring  basis? 

4.  How  does  this  measure  correlate  to 
aircraft  availability  in  terms  of  total  not  mission  capable 
(MICAP)  period  hours? 

5.  Do  CIP  managers  find  this  measure  useful? 
Investigative  Question  #2.  Does  measurement  of 

cycle  time  aid  the  Air  Force  in  managing  the  CIP 
effectively? 


managed 


aircraft 


Associated  Measurement  Questions; 

6.  What  is  the  variation  in  time  for  items 
in  the  CIP? 

7.  How  does  this  measure  correlate  to 
availability  in  terms  of  total  MICAP  period  hours? 

8.  Do  CIP  managers  find  this  measure  useful? 


SgoB.e 

The  scope  of  this  study  is  directed  at  validating 
measurements  for  performance  evaluation  of  the  Air  Force 
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CIP.  Due  to  availability  of  historical  critical  item 
program  data  and  the  currency  of  criteria  programmed  in 
WSMIS/GWAM,  only  Strategic  Air  Command  (SAC)  data  were 
collected.  Specific  weapon  systems  analyzed  within  SAC 
include  the  B-52,  B-1,  F-111,  C-135,  and  E-4  aircraft. 
Critical  item  data  were  collected  and  analyzed  beginning 
June  1990,  commensurate  with  the  latest  update  of  the  SAC 
WSMIS/GWAM  critical  item  candidate  selection  criteria. 

Assumptions  and  Limitations 

Though  data  collection  is  limited  to  SAC,  findings 
should  be  applicable  to  other  Air  Force  Major  Commands. 

This  assumption  is  based  on  the  similarities  existing 
between  MAJCOM  programs  and  the  problems  experienced  by 
each.  However,  due  to  limitation^  ut  access  to 
comprehensive  longitudinal  data,  conclusions  of  this  study 
are  subject  to  some  uncontrollable  bias.  Therefore,  this 
study  establishes  a  baseline  for  the  value  of  these 
measures,  but  does  not  provide  significant  statistical 
proof . 

Chapter  I  Summary 

The  purpose  of  the  Air  Force  CIP  is  to  provide  a 
proactive  manciement  tool  to  identify  and  improve  th^  status 
of  assets  that  are  most  degrading  to  weapon  system 
availability.  Numerous  and  recurring  problems  have 
prevented  this  program  from  accomplishing  that  goal  for  the 
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last  two  and  a  half  decades.  The  apparent:  reason  for  these 
shortfalls  is  the  lack  of  CIP  overall  management  controls. 

The  overall  objective  of  this  study  is  to  test  two 
measures  that  might  provide  a  means  of  evaluation  for  the 
CIP.  These  measures,  if  validated,  should  provide  an 
evaluation  of  the  current  status  of  the  program  and  a  means 
to  promote  continuous  improvement. 

Chapters  II  and  III  provide  detailed  insight  into  the 
background  of  this  issue  and  the  methodology  by  which  it  is 
approached  by  this  study.  Chapter  II  reviews  current 
literature  concerning  the  status  and  history  of  the  critical 
item  program  and  applicable  quality  control  measurement 
techniques  available.  Chapter  III  establishes  the  applied 
methodology  to  determine  conclusions  addressing  the  problem 
statement.  Chapter  IV  identifies  tne  findings  discovered 
through  implementation  of  the  stated  methodology.  Finally, 
Chapter  V  provides  the  conclusions  and  recommendations  based 
on  the  findings  from  Chapter  IV. 
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Literature  Review 


II . 

Introduction 

As  identified  in  Chapter  I,  numerous  studies  and  audits 
have  been  accomplished  addressing  the  critical  ibem  program 
(CIP).  These  studies  began  appearing  almost  at  the  same 
time  the  first  critical  item  management  program  was 
developed.  Though  numerous,  publication  of  these  studies 
have  not  followed  a  continuous  time-line.  Instead,  Air 
Force  concern  for  this  program  appears  to  come  in  strides  of 
great  interest  followed  by  periods  of  relaxed  attention. 

This  may  explain  why  the  CIP  still  contains  many  of  the 
problems  initially  identified  two  and  a  half  decades  ago. 

To  further  establish  the  basis  of  this  research,  this 
chapter  discusses  the  literature  available  on  all  aspects 
of  the  critical  item  program.  It  begins  with  a  review  of 
early  program  development,  including  initially  discovered 
problems.  A  chronological  development  details  the  important 
aspects  of  the  program  and  provides  the  reader  with  a 
logical  progression  resulting  in  its  current  status.  The 
history  of  the  critical  item  program  is  followed  by  review 
of  the  integration  and  improvement  of  new  automated 
information  system  capabilities  which  have  greatly  added  to 
its  performance.  Problems  identified  in  Chapter  I  are 
discussed  in  much  more  detail  in  this  chapter  to  ensure  full 
understanding  of  the  topic. 
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Also  important  to  this  research  is  a  review  of 
available  quality  control  literature.  The  selection  of 


measurement  instruments  is  of  key  importance  to  any 
scientific  research.  As  such,  a  review  of  quality  control 
measures  provides  the  reader  with  a  better  understanding  of 
the  methodology  used  by  this  study.  Therefore,  applicable 
statistical  quality  control  measures  are  also  reviewed. 

Critical  Item  Program  Development 

Placing  the  right  resources  at  the  right  place  when  they 
are  needed  has  been  the  logistic  objective  since  the 
cave  man  started  throwing  rocks.  (44:1) 

1967  to  1971.  The  Air  Force  Critical  Item  Program  (CIP) 

was  originated  in  September  1967  with  the  establishment  of 

Headquarters  Air  Force  Logistics  Command  Regulation  (AFLCR) 

67-21  (43:1).  Originally  this  program  was  called  the  High 

Not  Operationally  Ready  Supply  (NORS)  Critical  Item  Program 

(43:1).  The  time  frame  of  this  program's  development  fell 

around  the  midpoint  of  the  Vietnam  War.  As  it  still  is  now, 

the  goal  of  the  critical  item  program  was  to  increase  weapon 

system  availability  by  improving  the  asset  status  of  those 

items  most  contributing  to  a  weapon  system's  inability  to 

perform  its  mission.  With  weapon  system  availability  being 

of  significant  concern  during  a  war,  it  is  understandable 

why  this  program  received  so  much  attention  at  the  time. 

Just  prior  to  the  implementation  of  AFR  67-21,  several 

master's  theses  were  accomplished  in  this  management  area. 

In  1967,  Captains  Philip  Norton  and  Arthur  Tennyson 
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attempted  to  establish  two  objectives  concerning  critical 
item  forecasting.  They  were  successful  in  demonstrating 
their  first  objective,  that  causes  of  criticality,  directly 
related  to  changes  in  consumption  rates,  contributed  to  a 
significant  portion  of  items  in  a  critical  supply  status 
(40:86).  However,  their  second  objective,  to  develop  a 
mathematical  formula  to  identify  critical  items  by  comparing 
a  support  capability  to  the  actual  consumption  rate,  was 
unsuccessful  (40:87). 

Also  at  this  time  Mr.  Kenneth  Seifert  and  Captain  Martin 
Nakunz  addressed  a  similar  issue.  Their  objectives  were 
three  fold.  Their  first  objective  confirmed  that  Air  Force 
methodology  for  computing  demand  levels  of  repair  cycle 
assets  at  base  level  still  inflated  the  total  demand  levels 
for  Interchangeability  &  Substitute  (I&S)  Groups  (48:58). 
Their  second  objective  established  that  there  was  a  lack  of 
compatibility  in  reporting  systems  used  by  Air  Materiel 
Areas  (now  called  Air  Logistic  Centers)  and  the  base  level 
(48:59).  These  authors  maintained  that  "lack  of 
compatibility  casts  serious  doubt  upon  the  reliability  and 
integrity  of  NORS  requisitions  for  critical  items  within  the 
Air  Force  supply  network"  (48:59).  Their  third  objective 
determined  that  Air  Materiel  Areas  used  differing  approaches 
for  critical  item  management  (48:60).  Approaches  were  found 
significantly  different  for  the  management  of  reconc i 1 iat ion 
procedures.  Seifert  and  Nakunz  recommended  that  further 
study  be  accomplished  addressing  the  interface  of 


19 


requirement  determination  and  materiel  distribution  systems 
(48:61) . 

A  1968  study  by  Captains  Stanley  Condojani  and  Warren 
Welsh  accomplished  an  analysis  of  critical  items  of  cargo 
aircraft.  Their  two  objectives  were  first,  to  determine  the 
distribution  of  critical  items  with  respect  to  Federal 
Supply  Groups  (FSG);  and  second,  to  determine  if  there  was  a 
valid  technique  for  predicting  their  occurrence  (11:7).  For 
their  first  objective,  they  concluded  that  FSGs  16  (aircraft 
components  and  accessories)  and  66  (aircraft  instruments) 
comprise  68.3%  to  87.4%  of  all  cargo  aircraft  critical  items 
(11:78).  From  their  second  objective  they  determined  that  a 
correlation  between  critical  items  and  flying  hours  existed 
(11:78).  They  also  determined  that  the  higher  the  flying 
hours  the  better  this  correlation  becomes  (11:78).  Based  on 
their  conclusions,  they  recommended  a  mathematical  factor  be 
developed  for  requirements  computation.  The  factor  would  be 
a  function  of  expected  average  monthly  increase  in  flying 
hours  with  the  purpose  of  preventing  critical  item 
development  (11:79). 

In  1969  Ms  Florence  Phillips  and  Captain  Floyd  Hoo)cs 
evaluated  the  NORS  selection  criteria  for  the  selection  of 
High  NORS  critical  items  (43).  Their  study  was  based  on 
concerns  raised  by  both  audit  and  field  conclusions  that 
critical  item  selection  criteria  were  biased  in  their 
selection  of  critical  items.  The  criteria  of  specific 
concern  were  of  weapon  system  items  having  over  1000 


identified  NORS  hours  and  requiring  longer  than  60  days  to 
fill  back  orders  for  Uniform  Materiel  Movement  and  Issue 
Priority  System  (UMMIPS)  priority  codes  01  -  08  (43:41,52; 
4).  According  to  audit  and  field  conclusions,  the  criteria 
identified  items  for  critical  management  that  did  not  impact 
high  flying  hour  mission  aircraft,  while  simultaneously 
preventing  low  flying  hour  mission  aircraft  from  much  needed 
management  resources  (43:51).  Data  collected  by  Phillips 
and  Hooks  were  inconclusive  in  proving  their  hypotheses; 
however,  data  were  significant  enough  to  encourage  future 
studies  in  this  area. 

In  1971,  Captains  Raymond  Agnor  and  Douglas  Topping 
further  addressed  the  subject  of  criteria  bias  based  on 
weapon  system  fleet  size  (2:39).  Prom  their  study  they 
established  three  conclusions.  First,  by  comparing  fleet 
sizes,  they  determined  that  fleets  with  a  larger  population 
of  aircraft  had  a  lower  average  number  of  NORS  hours  per 
aircraft  than  did  smaller  population  fleets  (2:39).  Second, 
they  established  that  due  to  current  criteria,  items  with  a 
higher  average  number  of  NORS  hours  per  aircraft  were  not 
selected  as  critical  over  items  with  lower  averages  (2:45). 
Their  final  conclusion  found  "that  applying  a  weighted 
average  number  of  NORS  hours  per  aircraft  better  identifies 
those  items  most  adversely  affecting  mission  capability  of 
operational  aircraft  fleets”  (2:60).  As  a  result  of  these 
conclusions,  Agnor  and  Topping  recommended  further 
refinement  of  critical  item  selection  criteria  (2:70). 
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1982  to  1986.  As  mentioned  at  the  beginning  of  this 


chapter,  the  Vietnam  War  most  likely  influenced  the  degree 
of  interest  placed  on  the  critical  item  program  during  the 
late  1960s  and  early  1970s.  During  this  time  frame  at  least 
five  masters  theses,  one  audit,  and  an  unknown  number  of 
field  studies  were  accomplished  addressing  this  subject  area 
(40;  48;  11;  43;  2;  4).  However,  after  the  early  1970s,  a 
definite  lull  in  the  literature  for  critical  item  management 
took  place.  This  lull  can  presumably  be  attributed  to  a 
decrease  in  management  attention  to  the  subject.  It  is  not 
until  the  mid  1980s,  more  than  ten  years  after  the  above 
literature,  that  renewed  interest  in  this  area  is  found. 

In  1982,  an  Air  Force  Audit  Agency  report  identified 
more  deficiencies  in  critical  item  management  (5).  The 
audit  found  "numerous  Instances  of  critical  items  not  being 
intensively  managed,  while  erroneously  designated  critical 
items  were  being  afforded  special  attention"  (5:executive 
summary).  The  audit  also  revealed  that  items  in  the 
critical  item  program  took  times  ranging  from  four  to  twenty 
months  before  leaving  the  program  (5:4).  The  auditor 
concluded  that  quarterly  review  of  critical  items  by  Air 
Logistic  Centers  (ALC)  was  inadequate  to  improve  the 
readiness  posture  of  aircraft  (5:4).  The  report  recommended 
a  more  frequent  review  be  conducted  of  the  program.  Other 
conclusions  found  that  "uniform  procedures  were  not 
established  to  monitor  the  addition  and  deletion  of  items  to 
the  critical  item  program"  (5:6). 
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In  1985,  yet  another  masters  thesis  was  accomplished 

addressing  critical  item  performance.  Mr.  David  Thomson 

analyzed  the  critical  item  performance  of  the  Gardner- 

Dannenbring  Aggregate  Inventory  Model.  His  objective  was: 

to  determine  if  the  aggregate  model's  critical  item 
performance  can  be  improved  by  modifying  the  model  to 
discriminate  between  critical  and  non-critical  items  and 
compute  stockage  policies  which  will  concentrate  on 
reducing  the  number  of  back  orders  for  critical  items. 
(53:4) 

Based  on  Mr.  Thomson's  conclusions,  he  recommended  that: 

minimal  critical  item  back  orders  is  a  more 
operationally  oriented  supply  objective  than  variable 
costs  or  maximum  fill  rates  because  the  importance  of 
the  items  is  recognized  and  priority  is  given  to  those 
items  which  can  cause  an  aircraft  grounding.  (53:40) 

Mr.  Thomson's  study  was  the  first  to  examine  the  application 

of  a  specific  inventory  model  to  nuances  surrounding  the 

critical  it6m  program. 

A  1986  follow-up  audit  of  the  1982  audit  found  that, 
though  addressed,  1982  audit  recommendations  had  not  been 
adequately  resolved  by  management  (3:4).  The  findings  of 
this  audit  highlight  the  slow  improvement  process  that  the 
critical  item  program  has  demonstrated  from  its  inception. 

This  same  year  (1986),  the  Air  Force  Logistics 
Management  Center  (AFLMC)  performed  a  complete  study  of  the 
Air  Force  critical  item  management  program  (7).  The 
conclusions  and  recommendations  from  this  study  served  to 
revamp  the  entire  program  resulting  in  significant 
improvements.  Because  this  study  had  such  a  large  impact  on 
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management  of  the  current  critical  item  program,  its 
findings  and  conclusions  are  addressed  in  detail. 

This  study,  AFLMC  Project  LS841129,  was  conducted  based 
on  Major  Command  (MAJCOM)  and  ALC  requests  from  the  Fourth 
Critical  Item  Conference  (7:iii).  Concerns  of  these 
organizations,  reflected  the  same. problems  identified  by  the 
1982  and  1986  audits  referenced  above.  To  satisfy  these 
concerns,  AFLMC  identified  the  following  objectives  (7:1): 

1.  Review  and  evaluate  the  current  critical  item 

system. 

2.  Evaluate  possible  alternative  criteria  from 

identifying  critical  items. 

3.  Determine  who  should  identify  a  critical  item. 

4.  Recommend  standard  critical  item  criteria. 

5.  Recommend  changes  to  the  critical  item  system. 

In  addressing  the  objectives,  AFLMC  first  reviewed  the 
current  critical  item  program  criteria.  These  criteria,  as 
stated  above,  are  mainly  based  on  MICAP  hours  (hours 
attributed  to  a  weapon  system  not  operational  due  to  supply 
or  maintenance  actions,  originally  referred  to  as  NORS).  In 
this  review,  AFLMC  found  that  the  definition  of  a  MICAP 
incident  had  changed  several  times  from  1970  to  1982  (7:2). 
These  definitions  were  as  follows: 

1970  -  If  more  than  one  item  was  grounding  a  weapon 
system,  only  one  item  was  a  MICAP. 

1974  -  All  items  grounding  a  weapon  system  could  be 
called  a  MICAP. 

1976  -  Any  item  keeping  an  engine  unserviceable 
could  be  MICAP. 
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1977  -  Various  types  of  support  and  communications 

equipment  could  be  MICAP  reportable. 

1978  -  Any  item  with  a  MICAP  reportable  standard 

reporting  designator  (SRD)  could  experience  MICAP 
conditions . 

1982  -  The  Air  Logistics  Center's  Program  Depot 

Maintenance  lines  could  use  the  Force  Activity 
Designator  (FAD)  of  the  owning  command  for 
aircraft  and  an  item  preventing  a  functional 
flight  test  could  be  MICAP  if  the  flight  test  was 
scheduled  within  eight  days. 

These  changes  in  MICAP  definition,  though  not  affecting  the 
CIP  directly,  do  reflect  the  Air  Force's  increased  ability 
to  analyze  data  (7:2). 

In  their  analysis  of  the  system,  AFLMC  found  two 
problems  in  addition  to  those  identified  by  the  1982  audit 
(7:4).  The  first  problem  identified  the  current  system  as 
reactive.  Th's  /as  due  to  the  absence  of  a  system  to  detect 
potential  ..roblems.  The  second  problem  identified  the 
current  system  as  manually  workload  intensive.  Item 
managers  were  required  to  manually  accomplish  AFLC  Form  74, 
Critical  I tem/Warstopper  Data  for  each  item  attaining  the 
MICAP  hour  criteria.  This  form  required  extensive  research 
and  data  to  complete  properly,  creating  an  enormous  workload 
(7:4). 

AFLMC  also  collected  AFLC  and  Strategic  Air  Command 
(SAC)  MICAP  data  in  order  to  analyze  the  critical  item 
program  (7:4).  From  analysis  of  AFLC  data  they  discovered 
that  approximately  33%  of  managed  items  remained  critical 
for  over  one  year  (7:4).  In  addition  they  found  that  two  to 
three  percent  of  the  critical  items  managed  returned  to  the 
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program  within  one  year  after  having  initially  left  (7:4). 
From  analysis  of  SAC  data  they  discovered  that  83%  of  the 
items  meeting  critical  item  criteria  had  not  been  included 
in  the  critical  item  program  (7:5).  Failure  of  the  MAJCOMs 
to  include  such  items  in  the  critical  item  program  had  been 
a  point  of  contention  with  AFLC  for  some  time  (7:6). 

In  search  for  new  criteria,  AFLMC  tested  weapon  systems 
impact  ratio  as  a  possible  alternative  to  monthly  MICAP 
hours.  Weapon  systems  impact  ratio  (WSF)  i'  calculated  as 
follows : 

WSF  =  Total  MICAP  Hours  for  a  Stoc)t  Number _ 

Total  Possessed  Hours  for  the  Weapon  System 

In  their  analysis,  AFLMC  found  both  criteria  appropriate 

predictors  of  future  problem  items  (7:11).  However,  they 

also  found  that  the  MICAP  hour  criteria  was  biased  towards 

larger  population  aircraft  systems  while  the  WSF  criteria 

was  biased  towards  lower  population  aircraft  systems  (7:12). 

Based  on  a  review  of  the  past  literature  and  from  their 
first  hand  analysis,  AFLMC  determined  that  certain  selection 
and  management  criteria  must  be  attained  by  the  CIP  for  it 
to  be  successful  (7:17).  AFLMC's  selection  and  management 
criteria  were  as  follows: 

Selection  Criteria.  The  CIP  must  be  able  to; 

1.  Ran)«  items  according  to  weapon  system  impact. 

2.  Identify  pre-critical  categories. 

3.  Include  items  that  affect  sustainability  as 

well  as  readiness  and  awaiting  parts. 

4.  Use  more  than  one  dimension  to  select  items. 
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5.  Identify  the  overall  weapon  systems  impact. 


Management  Criteria.  The  CIP  must: 

1.  Reduce  the  manual  workload  associated  with 
evaluating  and  monitoring  critical  items. 

2.  Be  proactive;  it  must  be  able  to  identify  and 
prevent  items  from  becoming  critical. 

3.  Identify  different  categories  of  critical 
itemS/  so  that  items  that  cannot  get  well  in  the 
near  term  are  not  excessively  monitored. 

4.  Communicate  to  all  MAJCOMs  and  bases  what  items 
are  in  (or  not  in)  the  program,  "get  well"  actions, 
and  the  "get  well"  date. 

5.  Review  all  actions  to  improve  the 
supportabi 1 ity  of  the  item,  including  increasing 
the  worldwide  requirement. 

6.  Ensure  expediting  action  does  not  decrease  the 
capability  to  support  the  item  by  reducing  the 
worldwide  requirement. 

7.  Intensively  manage  all  items  that  severely 
impact  mission  support,  including  War  Readiness 
Materiel  shortages,  and  component  parts  generating 
awaiting  parts  (AWP)  conditions.  (7:17) 

Aware  that  modules  developed  for  the  Weapon  System 
Management  Information  System  (WSMIS)  would  improve  mass 
quantification  of  data,  AFLMC  recommended  a  three  tiered 
categorization  for  degraded  support  items  be  instituted  to 
address  the  above  requirements.  The  three  tiers  recommended 
from  least  to  most  critical  respectively  were:  potential 
problem,  problem,  and  critical  (7:20).  AFLMC  maintained 
that  early  identification  of  potential  problem  and  problem 
items  would  provide  a  proactive  capability  within  the 
program.  AFLMC  also  proposed  multidimensional  criteria  for 
distinguishing  items  among  the  three  tiers  (7:21).  AFLMC 
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emphasized  the  importance  of  multidimensional  criteria 
versus  critical  item  selection  based  on  only  one  data  point 
(7:22)  . 

Final  conclusions  and  recommendations  from  this  study 
include  (7:29): 

Conclus i ons . 

1.  In  the  past,  the  Air  Force  Critical  Item 
Program  has  been  reactive  and  relatively 
ineffective . 

2.  The  current  Critical  Item  Program  does  not 
consider  war  reserve  materiel  (sustainability) 
shortages  nor  does  it  consider  component  parts  that 
severely  impact  the  repair  of  end  items  for  weapon 
systems . 

3.  The, Critical  Item  Program  Selection  Criteria  is 
one-dimensional,  and  does  not  adequately  consider 
the  impact  on  a  weapon  system. 

4.  The  Automated  Critical  Item  Networlc  (ACIN)  and 
the  Weapon  Systems  Management  Information  System 
(WSMIS)  will  reduce  the  manual  wor)c  load  and  should 
significantly  improve  the  Critical  Item  Program. 

5.  Our  proposed  item  selection  criteria  is 
proactive,  multi -dimens ional ,  and  considers  an 
item's  impact  on  both  weapon  systems  readiness  and 
sustainability. 

6.  A  coding  scheme  to  identify  and  communicate 
item  manager  actions  to  the  MAJCOMs  should  improve 
the  management  of  critical  items. 

7.  Actions  taken  once  an  item  is  added  to  the 
Critical  Item  Program  may  reduce  pipeline  times, 
thereby  reducing  the  worldwide  demand  level  and 
decreasing  the  supportabi 1 i ty  of  that  item. 

Recommendations . 

1.  Continue  to  support  the  implementation  of  the 
Automated  Critical  Item  Network  (ACIN)  and  the 
Weapon  System  Management  Information  System 
(WSMIS ) . 
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2.  Service  test  AFLMC  proposed  critical  item 
selection  criteria  for  SAC  in  March  1987  as  part  of 
WSMIS. 

3.  Approve  AFLMC  proposed  critical  item  selection 
criteria. 

4.  Develop  a  coding  scheme  to  identify  and 
communicate  it:m  manager  actions  to  the  MAJCOMs. 

5.  Modify  the  Standard  Base  Supply  System  and  the 
Air  Force  Logistics  Command  requirements  system  to 
ensure  base  and  depot  repair  times  and  order  ship 
times  are  not  reduced  for  critical  items. 

Based  on  application  of  the  AFLMC  recommendations  and 
the  introduction  of  improved  automated  management 
information  system  capabilities,  today's  managers  possess 
potentially  powerful  management  tools.  Before  further 
describing  the  process  and  policy  of  the  current  critical 
item  program,  it  is  appropriate  to  describe  the  structure 
and  development  of  the  automated  critical  item  information 
systems  available  today. 


Automated  Information  Systems 

The  core  logistics  functions  of  Requirements, 
Acquisition,  Distribution,  and  Maintenance  are  so 
dependent  upon  information  systems  that  only  a  fraction 
of  the  normal  peacetime  business  could  be  conducted 
without  them.  (6) 

As  indicated  earlier,  the  most  significant  automated 
information  system  used  in  critical  item  management  is 
WSMIS.  WSMIS  was  created  to  support  wartime  objectives  and 
functions  of  Air  Force  Logistics  Command  (22:2-2) . 
Currently,  WSMIS  is  composed  of  four  operational  modules. 
These  modules  are  the  Readiness  Assessment  Module  (RAM), 
Sustainability  Assessment  Module  (SAM),  Requirements 
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Execution  Availability  Logistics  Module  (REALM),  and  the  Get 

Well  Assessment  Module  (GWAM)  (22:2-2).  The  three  modules 

used  in  critical  item  management  are  SAM,  RAM,  and  GWAM. 

SAM  projects  the  number  of  sorties  that  can  ^e  flown  and 
the  number  of  available  mission-capable  aircraft,  based 
on  wartime  tasking  and  logistics  resources.  SAM  uses 
operational  plans,  logistics  performance  factors  and 
logistics  resource  data,  and  other  Air  Force  data 
systems  to  assess  how  well  current  logistics  resources 
will  meet  combat  objectives.  SAM  identifies  those  items 
that  are  projected  to  prevent  weapon  systems  from 
achieving  their  wartime  objective  of  generating  sorties 
and  passes  this  information  to  GWAM.  (22:2-4) 

RAM  determines  the  readiness  of  the  weapon  system  and 
materiel  resources  required,  including  aircraft, 
engines,  and  support  equipment  to  conduct  wartime 
missions.  RAM's  objectives  are  to  provide  quantitative 
measures  of  readiness  as  close  to  real-time  as  possible. 
RAM  must  also  identify  the  readiness-limiting  items  so 
that  they  may  be  passed  to  GWAM  for  Get-Well  Plan 
development.  (22:2-4) 

GWAM  accepts  the  problem  parts  data  identified  in  RAM 
and  SAM,  along  with  information  from  other  data  systems, 
and  provides  the  IMS  with  the  information  to  develop 
solutions  for  these  problem  items.  GWAM  must  provide 
the  necessary  information  and  tools,  so  that  the  SPM, 

SCO,  and  IMS  may  make  educated  decisions  about  how  to 
manage  AFLC's  resources  and  provide  the  required  wartime 
support.  (22:2-4) 

The  current  WSMIS  operational  architecture  is  displayed  in 
Figure  1  (22:2-7). 

The  WSMIS  module  that  pulls  all  critical  item  program 
information  together  is  GWAM.  Using  SAM,  RAM,  and  various 
other  data  bases,  GWAM  is  responsible  for  tracking  all 
approved  critical  items.  The  overall  objectives  of  GWAM  are 
to  provide  all  levels  of  management  the  status  of  efforts  to 
solve  inadequate  logistics  support  of  items  (22:2-6).  This 
is  achieved  by  ensuring  the  specific  objectives  of  GWAM 
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Figure  1.  WSMIS  Operational  Modules  - 

(Figure  2)  are  accomplished.  These  specific  objectives  are 

1.  To  provide  an  automated  tool  to  support  the  CIP 

2.  Provide  on-line  access  to  Limiting  Factor  (LIMFAC) 
data  and  identify  those  that  are  most  critical 

3.  Identify  the  problems  that  created  the  LIMFAC 

4.  Aid  in  the  development  of  get-well  plans 

5.  Track  get-well  plan  implementation.  (22:2-6) 

GWAM  outputs  are  generated  from  all  four  sub-modules 
identified  in  Figure  2  (22:4-10).  These  outputs  include: 
the  Problem  Item  Summary,  the  Critical  Item  Summary,  the 
SAFETHACK  report,  the  Potential  Critical  Item  Listing,  the 
SAM  30  Day  Problem  Item  Listing,  the  Get-Well  Status 
Listing,  the  Location  of  Assets  Report,  the  Problem 
Indicators  Report,  and  the  Central  Leveling  Item  System 
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GWAM 


Figure  2.  GWAM  Overview  Diagram 


Report  (22:4-10/4-12).  A  short  summary  o£  each  o£  these 

reports  is  provided  below. 

Problem  Item  Summary.  Displays  the  current  problem 
items  from  the  four  file  sources:  Critical  Item, 
Potential  Critical  Item,  SAM  Wartime  Sustainability 
Problem  ItemS/  and  SAFETRACK  items  (22:4-10). 

Critical  Item  Summary.  Used  by  critical  item  management 
to  review  critical  item  workloads  and  to  monitor  status 
of  the  assigned  critical  items  (22:4-12). 

SAFETRACK .  Shows  the  Peacetime  Operating  Stock  (POS) 
levels  and  on-hand  quantities  (22:4-12). 

Potential  Critical  Item  Listing.  Lists  ranking  national 
stock  numbers  (NSN)  by  high  population  and  low 
population  engine  hours  (22:4-12). 

SAM  Day  30  Problem  Item  Listing.  Lists  the  current 
wartime  problem  items  by  reporting  organization  (22:4- 
12)  . 
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Get-Well  Status  Listing.  Lists  the  get-well  plan  status 
and  dates  for  MICAP,  priority  and  routine  Get-Well  dates 
(22:4-12) . 

Location  of  Assets  Report.  Incorporated  from  the  Ogden 
ALC  SAFE  Report,  it  generates  peacetime  asset  status 
(22:4-12) . 

Problem  Indicators  Report.  Is  a  composite  of  status  and 
requirement  information  about  a  requested  NSN  (22:4-12). 

Central  Leveling  Item  System  Report.  Is  the  on-line 
version  of  the  CLIS  report  available  in  hard  copy  from 
the  D028  system  (22:4-12). 

These  reports  provide  managers  with  substantial  information 
about  individual  item  status  and  Indicate  potential  and 
existing  problem  areas. 

In  addition  to  the  above  reports,  GWAM  provides  the  user 
with  on-line  access  to  automated  Air  Force  Form  74,  Critical 
Item/Warstopper  Data  (22:2-21).  This  capability  allows  the 
user  to  view  the  status  of  any  critical  item  with  an 
approved  get-well  plan.  As  MAJCOMs  play  a  key  role  in 
critical  item  identification  (described  in  detail  later  in 
this  chapter),  the  capability  to  remotely  view  this 
information  in  real-time  is  invaluable. 


Current  Criteria 


An  ever-increasing  reliance  upon  machines,  self- 
propelled  weapons,  and  highly  sophisticated  weapon 
systems  places  increasing  importance  upon  effective 
supply  support.  An  aircraft  that  won't  fly,  a  nuclear 
field  weapon  that  won't  fire,  or  a  missile  on  a  launch 
pad  that  is  not  capable  of  performing  its  intended 
mission  is  valuable  only  to  the  enemy.  (1:3) 

AFLMC  recommendations  brought  about  the  institution  of  a 

three  tiered  categorization  for  degraded  support  items. 

Though  the  labels  and  specific  criteria  of  the  current 
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system  are  slightly  different  from  AFLMC's  recommendations, 
their  basic  concepts  have  been  put  into  operation. 

According  to  AFM  67-1,  Vol  1,  Part  One,  Chapter  26,  Air 
Force  Critical  Item  Program,  the  lowest  tier,  "Problem," 
identifies  items  causing  minor  limitations  to  weapon  system 
operations  (15:26-9).  The  second  tier,  "Potential 
Critical,"  identifies  those  items  that  restrict  weapon 
system  operations,  but  not  to  the  extent  critical  items  do 
(15:26-9).  The  final  tier,  "Critical,"  identifies  those 
items  having  the  most  detrimental  impact  on  mission 
performance  (15:26-9).  Use  of  these  three  hurdles  is  the 
basis  for  the  critical  item  selection  process. 

Critical  Item  Selection  Process.  To  better  understand 
the  critical  item  selection  process,  this  section  will 
describe  the  individual  functional  areas  and  their 
role/responsibility  in  the  critical  item  management  process. 

Air  Logistic  Centers  (ALC).  The  ALCs  are 
responsible  for  identifying  major  causes  of  support 
problems,  talking  required  corrective  action,  and  ensuring 
all  associated  administrative  responsibilities  are  completed 
(15:26-5).  In  addition,  they  are  to  establish  a  Critical 
Item  Review  Committee  and  appoint  an  ALC  Critical  Item 
Monitor  to  regularly  review  and  administer  the  local 
critical  item  program  (15:26-5). 

Major  Commands  (MAJCOM).  Each  MAJCOM  establishes 
its  own  unique  criteria  using  the  guidelines  identified 
above.  Having  done  this,  MAJCOMs  are  responsible  for 
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reviewing  the  CIP  items  to  ensure  their  legitimacy  within 
their  own  criteria  (15:26-5).  Along  with  the  ALC  System 
Program  Manager  (SPM),  the  MAJCOMs  then  complete  a  review 
and  negotiation  of  the  items  they  have  submitted  for  final 
inclusion  in  the  CIP  (15:26-5). 

System  Program  Manager  (SPM).  The  SPM  acts  as  the 
point  of  contact  for  all  critical  hurdle  items  affecting  the 
weapon  system  (15:26-6).  In  doing  this,  the  SPM  reviews 
automated  Forms  74  (Get  Well  Plans)  and  the  MAJCOM 
identified  critical  items  to  recommend  changes  where 
appropriate  (15:26-6).  The  SPM  must  also  coordinate  with 
other  SPMs  and  the  MAJCOMs  concerning  funding  strategies  for 
corrective  actions.  Important  in  this  process  is  the  SPMs 
participation  in  the  MAJCOM  negotiation  for  addition  and 
deletion  of  items  to  the  CIP  (15:26-7). 

ALC  Critical  Item  Manager  (CIM).  The  CIM  acts  as 
the  point  of  contact  between  the  ALC'  and  Air  Force  Logistics 
Command  (AFLC)  (15:26-5).  The  CIM  is  a  permanent  member  of 
the  ALC  Critical  Item  Review  Committee  with  the 
responsibility  of  monitoring  CIP  corrective  actions,  and 
ensuring  automated  Form  74  (GET  Well  Plans)  are  managed 
properly  (15:26-5). 

Item  Management  Specialist  (IMS).  The  IMS  is 
responsible  for  core  logistics  functions  including 
requirements  budgeting,  distribution,  and  transportation 
(15:26-7).  To  accomplish  this,  the  IMS  coordinates 
expedited  repair  actions  with  the  production  management 
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specialist,  tracks  worldwide  spare  assets,  develops 
automated  Forms  74  (Get  Well  Plans),  and  initiates  emergency 
procurement  requests  (15:26-7). 

Equipment  Specialist  (ES).  The  ES  is  responsible 
for  identifying  needed  reliability  and  maintainability 
improvements  (15:26-7).  The  ES  accomplishes  this  by 
reviewing  automated  Forms  74  no  later  than  the  15th  day  of 
each  month  (15:26-7). 

Production  Management  Specialist  (PMS).  The  PMS 
identifies  all  cases  where  negotiated  repair  actions  are 
less  than  required  repair  actions  (15:26-7).  This 
highlights  those  items  that  may  be  additionally  detained  due 
to  maintenance  capability.  Like  the  ES,  the  PMS 
accomplishes  this  task  by  a  review  of  the  automated  Form  74 
(15:26-7). 

ALC  Critical  Item  Review  Committee.  This  committee 
is  chaired  by  the  Materiel  Manager  (MM)  and  composed  of  two 
letter  directorate  level  representatives  (15:26-8).  This 
committee  is  required  to  meet  monthly  to  review  and 
determine  appropriate  corrective  actions  for  all  newly 
assigned  critical  items  and  those  recurring  for  four 
consecutive  months  (15:26-8).  In  accomplishing  this  review, 
SPM,  IMS,  PMS,  and  ES  personnel  will  attend  these  meetings 
as  needed  to  provide  required  information. 

Critical  Item  Selection.  Current  guidance  requires 
critical  item  program  selections  be  based  on  one  or  more  of 
the  multidimensional  criteria  identified  below  (15:26-8). 
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As  stated  earlier,  individual  MAJCOMs  are  responsible  for 
tailoring  specific  guidelines  within  these  criteria  to 
establish  critical  item  selection  (15:26-5). 

1.  Weapon  System  Availability:  Items  that  limit 
weapon  system  performance  are  identified  based  upon 
their  limitation  to  predetermined  aircraft  goals 
established  for  each  MDS  by  each  MAJCOM.  This 
determination  is  accomplished  analytically  in  the 
Weapon  System  Management  Information  System 

( WSMI S ) /Sustainabi 1 i ty  Assessment  Module  (SAM). 

2.  Weapon  System  Factors:  Is  the  total  number  of  non 
mission  capable  (MICAP)  hours  generated  by  an 
individual  line  item  divided  by  the  total  possessed 
hours  of  the  weapon  system  that  the  item  supports  for 
the  most  current  month  available. 

3.  Monthly  MICAP  Hours:  An  established  total  of  MICAP 
hours  for  the  previous  month. 

4.  Monthly  MICAP  Incidents:  An  established  number  of 
MICAP  incidents  for  the  previous  month. 

■5.  Yearly  MICAP  Incidents:  An  established  number  of 
MICAP  incidents  for  the  previous  twelve  months. 

6.  Yearly  Awaiting  Parts  (AWP)  Incidents:  An 
established  number  of  AWP  incidents  for  the  previous 
twelve  months. 

7.  Single  Point  Failure:  Items  which  have  zero  MICAP 
tolerance  due  to  mission  impact. 

8.  Other  Parameters  Developed  by  MAJCOMs. 

9.  Ensure  supply  assistance  follow-up  procedures 

comply  with  prescribed  directives  AFM  67-1,  Vol  1,  Part 
One,  Chapter  1,  Section  E  and  Vol  2,  Part  Two,  Chapter 
9,  Paragraph  110.  (15:26-8) 

All  criteria  provide  a  quanitfiable  means  to  evaluate 
critical  items  with  the  exception  of  number  eight.  The 
eighth  criterion  allows  the  MAJCOMs  to  apply  any  unique 
hurdle  criteria  they  deem  appropriate  in  determining 
critical  items.  Currently,  there  is  no  requirement  for  the 
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MAJCOMs  to  justify  these  unique  criteria  (36).  This 

ability,  along  with  their  ultimate  authority  to  determine 

what  is  included  and  excluded  from  the  program,  gives 

MAJCOMs  enormous  control  over  CIP  management. 

Important  in  the  critical  item  management  process  is  the 

determination  of  when  an  item  should  leave  the  program. 

When  an  item's  asset  status  has  been  sufficiently  corrected, 

the  item  is  termed  "well"  (15:26-10).  AFM  67-1,  Vol  1,  Part 

One,  Chapter  26,  defines  well  as  follows: 

Economic  Order  Quantity  (EOQ)  Items.  Items  not  having 
met  one  of  the  standard  thresholds  for  the  previous 
three  months  and  no  priority  01-08  back  orders  at  the 
time  of  the  review.  Items  must  be  supportable  through 
on-hand  stock  and  or  firm  due-ins  from  procurement. 
(15:26-10) 

Recoverable  Items.  Items  not  having  met  one  of  the 
standard  thresholds  for  the  previous  three  months  or  no 
priority  01-08  back  orders  exist  at  the  time  of  the 
review.  Items  must  be  supportable  through 
redistribution  of  assets,  repair  or  firm  due-ins  from 
procurement  with  delivery  within  the  next  30  days. 
(15:26-10) 

Upon  achievement  of  the  above  two  criteria,  critical  items 
are  to  be  deleted  from  the  CIP.  In  addition,  critical  items 
may  also  be  deleted  upon  negotiation  between  the  MAJCOM  and 
SPM  (15:26-10). 


Aircraft  Availability 

Aircraft  availability  is  a  concept  which  attempts  to 
link  supply  purchase  actions  to  operational  capability 
(41:36).  From  a  supply  perspective,  aircraft  availability 
is  based  on  the  number  of  aircraft  awaiting  and  not  awaiting 
a  resupply  action,  independent  of  maintenance  or  crew 
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requirements  (41:36).  To  further  demonstrate  this  concept, 
an  aircraft  is  considered  to  be  available  if  all  parts  have 
been  provided,  regardless  of  pending  maintenance  actions  or 
crew  shortfalls.  Under  this  concept,  spares  are  purchased 
to  maximize  aircraft  availability  measures  verses 
maximization  of  commonly  used  aircraft  spares  fill  rates 
(41:37).  Benefits  of  using  the  aircraft  availability  model 
(AAM)  is  that  it  purchases  only  those  items  affecting 
aircraft  availability,  thus  avoiding  unnecessary  inventory 
and  waste  of  limited  resources  (41:37).  Given  the  CIP  is  a 
key  means  by  which  resupply  actions  are  undertaken,  it  is 
reasonable  to  conclude  that  problems  affecting  the  CIP  also 
impact  the  measure  of  aircraft  availability. 

Summary  of  the  Critical  Item  Program 

As  has  been  presented,  the  critical  item  program  is  a 
dynamic  management  process  crucial  to  the  effective  and 
efficient  support  of  Air  Force  weapon  systems.  It  is  a 
program  designed  to  enable  proactive  administration  of 
problem  support  items  in  order  to  prevent  management 
reactions.  The  primary  goal  of  the  CIP  has  been  to  increase 
weapon  system  availability.  Having  originated  at  the  height 
of  the  Vietnam  War,  it  is  a  process  that  has  been  in 
development  for  over  two  and  a  half  decades.  During  this 
time,  the  program  has  undergone  numerous  changes  and 
improvements  culminating  in  the  program  used  today. 

However,  in  this  process,  the  program  has  suffered 
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significant  and  recurring  difficulties  that  have  prevented 
it  from  achieving  its  full  potential. 

Close  inspection  of  these  difficulties  have  revealed 
several  contributing  factors.  These  factors  include 
incorrect  critical  item  selection  criteria,  conflicting 
regulatory  guidance,  insufficient  management  attention, 
and  reactionary  management  practices  (12;  36;  15;  7:29). 
However,  more  significant  than  these  shortfalls  is  the  clear 
lack  of  a  process  for  measuring  the  performance  of  the 
overall  program.  Because  of  this  shortfall,  a  means  by 
which  to  satisfy  the  operational  deficiencies  of  the  program 
is  unavailable.  As  a  result,  critical  item  managers  at  all 
levels  must  do  the  best  they  can  to  improve  a  program  for 
which  they  have  no  measure  for  how  well  it  is  performing. 
Based  on  these  circumstances,  it  becomes  apparent  that  a 
process  for  measurement  of  the  program  is  necessary  to 
establish  goals  common  to  all  critical  item  management 
activities.  Without  these  common  goals,  differences  of 
opinion  and  the  resulting  problems  will  continue  to  exist. 
Currently,  AFLC  and  MAJCOM  managers  differ  significantly  in 
their  individual  concerns  addressing  CIP  management  actions. 
Because  their  respective  approaches  have  little  in  common, 
actions  necessary  to  improve  the  program  have  been  difficult 
to  negotiate  between  the  two  sides  (36).  As  a  result,  the 
process  of  improvement  has  been  further  impaired.  Sink  and 
Tuttle  clearly  state  the  need  for  measurement  as: 
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Measure  to  improve.  Measure  to  provide  your  management 
team  with  new  insights  into  why  the  system  performs  the 
way  it  does,  where  it  can  be  improved,  and  when  the 
system  is  in  control  or  out  of  control.  (49:1) 

According  to  Harrington,  "Measurement  is  at  the  heart  of  any 

improvement  process... If  something  cannot  be  measured,  it 

cannot  be  improved"  (28:19).  Simply  stated,  without 

measurement  of  the  total  system,  the  quality  of  its  product 

is  unknown  and  therefore  uncontrollable.  To  address  the 

question  of  measurement  and  how  best  to  approach  it,  a 

review  of  quality  control  techniques  is  necessary. 

Quality  Control 

Some  processes  in  nature  exhibit  statistical 
contr ol . . . But  a  state  of  statistical  control  is  not  a 
natural  state  for  a  process.  It  is  instead  an 
achievement,  arrived  at  by  elimination  one  by  one,  by 
determined  effort,  of  special  causes  of  excessive 
var iat ion .  (13:5) 

Quality  control  is  a  concept  that  has  been  around  many 
years.  Some  would  argue  that  quality  control  has  existed 
since  the  first  craftsman.  In  the  context  that  this  study 
addresses  quality  control  management,  the  first  documented 
source  dates  back  to  the  early  1920s.  In  1924,  Dr.  Walter 
Shewart  first  introduced  statistical  quality  control  at  the 
Bell  laboratories  (39:102).  Dr.  Shewart  established  the 
concept  that  quality  could  be  tracked  and  controlled  using 
statistical  observation  (39:102).  He  did  this  through 
defining  the  limits  of  random  variation  and  setting 
acceptable  upper  and  lower  limits  so  that  outliers  could  be 
easily  detected  and  the  causes  studied  (55:7). 
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A  student  of  Dr.  Shewart's,  Dr.  Edward  Deming,  put  these 
concepts  to  work  during  World  War  II  to  aid  the  government 
in  wartime  procurement  (55:8).  Later  in  1947,  Di  Ipiu^iig’s 
skills  were  put  to  use  in  preparing  the  1951  Japanese  census 
(55:10).  Through  his  activities  in  Japan,  Dr.  n''-.l..g's 
ideas  and  teachings  of  Shewart's  techniques  became  very 
popular  with  Japanese  businessmen  (55:12).  While  America 
was  not  concerned  with  quality  due  to  overwhelming  world 
demand,  Japan  was  very  interested  due  to  the  need  to  rebuild 
its  industrial  base.  Through  the  years  Japan's  central 
tenet  in  quality  control  had  been  to  emphasize  gradual 
process  improvement  (19:580).  This  Japanese  commitment  to 
quality  and  continuous  improvement  have  placed  them  in  the 
position  they  are  today. 

As  a  result  of  the  unparalleled  success  of  Japanese 
industry  in  the  last  several  decades,  quality  control  has 
become  more  than  just  a  popular  business  buzz  word.  For 
American  industry,  the  issue  of  quality  has  become  a  matter 
of  survival.  In  today’s  business  environment,  a  failure  of 
a  corporation  to  recognize  this  situation  can  have 
devastating  results.  The  intense  budgeting  pressures  on  the 
Department  of  Defense  (DOD)  have  also  made  quality  a  concern 
in  the  public  sector. 

In  a  1989  article.  General  Hansen,  then  Commander  of  Air 
Force  Logistics  Command,  stated  "30'’o  of  the  quality  problems 
we  have  experienced  are  due  to  process  deficiencies" 

(26:12).  Thus,  in  review  of  quality  issues  government  must 
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ask  itself  two  questions:  "is  government  doing  the  right 
things... and  is  government  doing  it  right?"  (46:16). 
Questions  such  as  these  and  the  answers  that  have  proceeded 
them  have  lead  to  the  DOD's  adoption  and  commitment  to  Total 
Quality  Management  (TQM)  (42:18).  The  resulting  dominant 
theme  in  the  DOD's  TQM  program  is  commitment  to  quality  and 
continuous  improvement  (42:18).  To  better  understand  what 
this  means,  a  brief  description  on  quality  doctrine  and  some 
basic  definitions  are  useful. 

Statistical  Quality  Control.  The  purpose  of  statistical 
quality  control  (SQC)  is  to  improve  the  product.  SQC 
strives  to  prevent  defects  by  gathering  and  analyzing  data 
in  order  to  identify  problems  (25:19).  Though  opponents  to 
quality  control  argue  that  the  process  costs  too  much  money, 
the  opposite  is  usually  the  case.  Gilmore  notes  that 
"improving  product  or  service  quality  can  lead  to  steep 
short  run  reductions  in  quality  expenditures  and  to 
substantial  long-run  reductions  in  total  operating  costs" 
(23:21).  It  logically  follows  that  continuous  improvement 
will  persist  in  reducing  operating  costs.  Gilmore  defines 
continuous  improvement  as  "the  integration  of  organizational 
philosophy,  techniques,  and  structure  to  achieve  sustained 
performance  improvements  in  all  activities  on  an 
uninterrupted  basis"  (23:21).  This  definition  identifies 
that  quality  control  is  not  only  implementation  of  measures, 
but  also  the  complete  dedication  to  the  concept  by  all 
i nvolved . 
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As  introduced  above,  SQC  (also  referred  to  as 
statistical  process  control)  is  the  observation  of  random 
variation  within  acceptable  limits  in  order  to  identify 
outliers  (55:7).  This  variation  can  be  broken  down  into  two 
types,  chance  (common  and  numerous  but  which  are  not 
feasible  to  detect  or  identify)  and  assignable  (a  factor 
which  contributes  to  variation  and  can  be  detected 
reasonably )( 39 : 104 ) .  It  is  the  latter,  assignable 
variation,  with  which  SQC  is  concerned. 

To  evaluate  this  variation  there  are  two  categories  of 
measures,  variables  and  attributes  (24:3).  A  variable  is 
present  "when  a  record  is  made  of  an  actual  measured  quality 
characteristic,  such  as  a  dimension  expressed  in  thousandths 
of  an  inch"  (24:3).  An  attribute  is  present  "when  a  record 
shows  only  the  number  of  articles  conforming  and  the  number 
of  articles  failing  to  conform  to  any  specified 
requirements"  (24:3).  The  presence  of  an  attribute  or  a 
variable  will  determine  the  best  SQC  measurement  device  to 
use.  For  example,  Shewart's  "xR"  chart  (all  charts  are 
discussed  later  in  this  chapter)  is  best  suited  to  variable 
measurement,  while  Shewart's  "p"  chart  is  best  suited  to 
attribute  measurement  (24:4). 

To  implement  these  concepts,  a  measure  must  txist  upon 
which  to  base  management  controls.  Juran  defines  the  ideal 
unit  of  measure  as  one  which: 
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Provides  an  agreed  basis  for  decision  making... is 
understandable. . .applies  broadly... is  susceptible  of 
uniform  interpretation ...  is  economic  to  apply... and  is 
compatible  with  existing  designs  of  sensors.  (32:76-78) 

Before  further  addressing  the  two  specific  measures 

contained  in  this  study,  a  brief  review  of  the  types  of  data 

and  purposes  of  data  collection  is  appropriate. 


and  Purj 


Data  Collection, 


order 


measure  something,  data  must  first  be  collected.  How  the 


data  are  collected  will  determine  which  measurement 


techniques  are  appropriate.  Ishikawa  classifies  data 
collection  into  five  categories  (31:1-2)  as  follows: 


Data  to  assist  in  understanding  the  actual  situation. 
These  data  are  collected  to  check  the  extent  of  the 
dispersion  in  part  sizes  coming  from  a  machining 
process,  or  to  examine  the  percentage  of  defective  parts 
contained  in  lots  received. 


Data  for  analysis.  Analytical  data  may  be  used,  for 
example,  in  examining  the  relationship  between  a  defect 
and  its  cause.  Data  are  collected  by  examining  past 
results  and  making  new  tests. 

Data  for  process  control.  After  investigating  product 
quality,  this  kind  of  data  can  be  used  to  determine 
whether  or  not  the  manufacturing  process  is  normal. 
Control  chars  are  used  in  this  evaluation  and  action  is 
taken  on  the  basis  of  these  data. 

Regulating  Data.  This  is  the  type  of  data  used,  for 
example,  as  the  basis  for  raising  or  owering  the 
temperature  of  an  electric  furnace  so  that  a 
standardized  temperature  level  may  be  maintained. 

Actions  can  be  prescribed  for  each  datum  and  measures 
taken  accordingly. 

Acceptance  or  rejection  data.  This  form  of  data  is  used 
for  approving  or  rejecting  parts  and  products  after 
inspection.  There  are  two  methods:  total  inspection  and 
sampling . 
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Additionally,  Ishikawa  identifies  two  general  kinds  of  data 
(31:3)  : 

Measurement  data;  continuous  data  (length,  width, 

time,  etc.  ) 

Countable  data:  enumerate  data  (number  of 

defectives,  number  of  defects, 
percentage  defective,  etc.) 

Percent  Defects  and  Cycle  Time.  The  two  measures 
considered  in  this  study  encompass  both  of  Ishikawa's 
measurement  types.  Percent  defects  falls  under  countable 
data  and  cycle  time  under  measurement  data.  In  addition, 
these  measures  fall  under  Juran's  third  and  first  species  of 
measures  respectively  (see  Table  1).  Juran's  third  species 
of  measure  is  associated  with  errors  and  failures  (percent 
defects),  while  his  first  and  second  species  of  measures 
addresses  the  technological  level  and  product  performance 
(cycle  time ) ( 32 : 72 ) . 

Recent  applications  of  these  measures  have  enabled  two 
American  corporations  to  achieve  exemplary  levels  of 
quality.  Specifically,  these  companies  were  Motorola  and 
Xerox.  Due  to  the  successful  application  of  these  measures 
within  private  industry.  Government  interest  has  begun  to 
develop  surrounding  their  use  (16). 

William  Smith,  Vice  President  for  Quality  of  Motorola, 
claims  their  company  useu  these  measures  to  instill  the 
concept  of  quality  into  all  workers  versus  those  just  in  the 
quality  department  (16:35).  Smith  further  claims  these 
measures  are  generic  in  nature,  in  addition  to  being  easily 
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Table  1.  Juran's  Six  Species  of  Measures  (32:72-73) 


First  Species,  Technological  Level: 

QUALITY  FEATURE _ UNIT  OF  MEASURE _ 

Distance . Kilometers/  miles 

Weight . Grams,  ounces 

Time . Hours,  minutes 

Electrical  current . Amperes 

Temperature . Degrees 

Other  technological  units  of  measure  are  unfamiliar 
to  most  laymen  but  are  well  known  to  the 
technologists . 


Second  Species.  Product  Performance: 

QUALITY  FEATURE _ UNIT  OF  MEASURE _ 

Fuel  efficiency . Miles  per  gallon 

Kilometers  per/liter 
Timeliness  of  service. . .Minutes,  hours,  days 
Continuity  of  service .. .Percent  "uptime" 


Third  Species.  Errors  and  Failures: 

QUALITY  FEATURE _ UNIT  OF  MEASURE _ 

Defect  content . Percent  defective 

in  goods  demerits  per  unit 

Field  failures . Mean  time  between 

failures,  maintenance 
hours  per  thousand, 
operating  hours 

Service  interruption. .. .Percent  downtime 

Error  content  of . Percent  error  in 

services  billing 


Fourth  Species,  Functional  Department  Performance: 

FUNCTIONAL  DEPARTMENT  EXAMPLE  OF  MEASURE 

Product  development . Months  required  to 

launch  new  products 

Purchasing . Cost  of  poor  quality 

per  dollar  of 
purchases 

Manufacture . Cost  of  poor  quality 

per  dollar  of 
manufacturing  cost 


47 


Table  1  (continued). 

Fifth  Species,  Upper  Management  Level: 

QUALITY  FEATURES _ UNITS  OF  MEASURE 

Competitiveness  in . Ratio  of  products  per- 

the  market  place  formance  to  that  of 

leading  competitors 

Cost  of  poor  quality . Ratio  of  cost  of  poor 

quality  to  sales 


Sixth  Species.  Evaluation  of  Managers  Performance: 

This  species  of  measure  is  a  combination  of 
departmental  quality  performance  and  specific 
deeds  done. 

understood.  This  makes  them  very  applicable  throughout  many 
quality  control  situations  (50).  Backing  Smith's  claims  is 
the  company's  recent  achievement  of  the  Baldrige  Award  for 
quality  excellence.  Motorola  has  reduced  percent  defects  to 
300  failures  per  million  units  ( . 0003 ) ( 37 ; 108 ) .  Motorola's 
goal  is  to  continue  to  reduce  this  number  to  3.4  failures 
per  million  by  1992  (37:108).  Motorola  has  also  shown  vast 
improvements  in  cycle  time,  such  as  reduction  of  portable 
radio  fill  rates  from  55  to  15  days  (37:108).  Again, 
striving  for  continuous  improvement.  Motorola  hopes 
to  reduce  this  to  7  days  in  the  near  future  (37:108). 

Xerox  Corporation  is  another  recent  winner  of  the 
Baldrige  Award,  and  they  too  have  used  the  measure  of 
percent  defects  to  improve  product  performance.  Xerox’s 
goal  for  the  1990s  is  125  defects  per  million  with  the 
future  goal  being  zero  defects  (37:104).  In  short,  these 
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two  American  corporations  have  proven  SQC  can  be  used 
effectively  in  the  United  States. 

Measurement  Charts.  Having  described  the  basis  and 
doctrine  of  SQC,  in  addition  to  establishing  the  past 
validity  for  the  measures  of  percent  defects  and  cycle  time, 
one  must  now  consider  the  measurement  charts  available  upon 
which  variation  can  be  monitored.  Looking  at  the  quality 
literature,  one  finds  that  numerous  measurement  charts  are 
available.  To  justify  the  measurement  charts  used  in  this 
study  (further  presented  in  Chapter  III),  several  quality 
control  charts  are  presented  along  with  their  individual 
contr i but i ons . 

Pareto  Analysis.  Around  the  turn  of  the  century 
(1897),  V.  Pareto,  an  Italian  economist  identified  in  a 
formula  that  the  distribution  of  income  is  unevenly 
distributed  with  most  of  the  income  held  by  a  very  small 
population  of  people  (35:18).  Dr.  J.M.  Juran  applied  this 
concept  in  classifying  quality  control  problems  and  named  it 
Pareto  Analysis  (35:18).  Visually,  a  pareto  chart  is  very 
similar  to  a  vertical  bar  graph  (9:17).  The  purpose  of  the 
pareto  chart  is  to  identify  priorities  (55:105)  or  determine 
which  problems  are  most  significant  (7:155).  Figure  3 
displays  an  example  of  a  pareto  chart.  By  graphing 
the  errors  or  defects,  managers  can  easily  identify  those 
areas  causing  the  most  problems  and  address  them  first 
(31:43).  Pareto  charts  are  often  used  as  the  first  step 
in  problem  identification  (31:46). 
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PARETO  CHART 
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M2  P-111  C-135  B-1 

WEAPON  SYSTEM 


Figure  3.  Pareto  Chart 


Histograms .  According  to  Scherkenbach,  histograms 
are  one  of  the  most  misused  statistical  tools  as  they  are 
only  effective  if  measuring  a  process  within  statistical 
control  (47:106).  Basically,  a  histogram  is  useful  in 
identifying  how  frequently  something  is  observed  or  occurs 
(55:109).  Harrington  describes  a  histogram  as  "A  type  of 
bar  graph,  used  to  display  distribution  of  whatever  is  being 
measured”  (27:206).  In  short,  "A  histogram  reveals  the 
amount  of  variation  that  any  process  has  within  it"  (9:36). 
Figure  4  provides  an  example  of  a  histogram. 

Cause  &  Effect  Diagram.  The  cause  and  effect 
diagram  was  introduced  in  1953  by  Dr.  Kaoru  Ishikawa  in 
summarizing  the  opinions  of  engineers  towards  quality 
(35:26).  "Cause  &  Effect  Diagrams  (fishbone  diagrams)  are 
drawn  to  illustrate  the  various  causes  affecting  a  process 
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by  sorting  out  and  relating  the  causes”  (31:25;  9:24-25). 

The  Japanese  Industrial  Standards  define  cause  and  effect 
diagrams  as  ”a  diagram  which  shows  the  relation  between  a 
quality  characteristic  and  factors"  (35:26).  Use  of  cause 
and  effect  diagrams  are  usually  an  initial  step  by  which 
management  identifies  the  causal  factors  effecting  a  process 
(31:18).  Figure  5  is  an  example  of  a  cause  and  effect 
diagram. 

Scatterqram.  A  scattergram  charts  relationships 
between  two  variables  (55:111).  Brassard  maintains  it  is 
most  applicable: 

When  you  need  to  display  what  happens  to  one  variable 
when  another  variable  changes  in  order  to  test  a  theory 
that  the  two  variables  are  related.  (9:44) 

This  is  useful  in  identifying  cause  and  effect  relationships 

among  the  two  variables  observed.  In  determining  these 
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Figure  5.  Cause  and  Effect  Diagram 


relationships,  a  correlation  can  be  evaluated  (31:92).-  If 
the  trend  of  the  observations  is  upward  to  the  right,  a 
positive  correlation  exists;  if  the  trend  is  downward  to  the 
right,  a  negative  correlation  exists;  if  there  is  no  trend 
visible  there  is  no  correlation  (31:90-91).  Figure  6 
displays  an  example  of  a  scattergram  with  positive 
correlation . 

Control  Charts.  These  charts,  the  most  commonly 
used,  were  introduced  by  Dr.  Shewart  in  1924  (8:249;  35:92). 
According  to  Deming: 

The  control  chart  sends  statistical  signals,  which 
detect  existence  of  a  special  cause,  or  tell  us  that  the 
observed  variation  should  be  ascribed  to  common  causes, 
chance  variation  attributable  to  the  system.  (14:319) 
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Figure  6.  Scatter  Diagram 


Basically  there  are  two  kinds  of  control  charts, 
continuous  value  and  discrete  value  (35:94).  Within  these 
two  kinds  there  are  several  variations  which  are  described 
in  Table  2.  Variables,  actual  measured  quality 
characteristics,  are  most  appropriately  evaluated  using 
continuous  value  charts  (24:3;  35:94).  Attributes,  the 
number  of  articles  conforming  or  nonconforming  to  specified 
requirements,  are  most  appropriately  evaluated  using 
discrete  value  charts  (24:3;  35:94).  To  further  illustrate 
the  usefulness  of  these  charts,  the  p  and  xR  chart  are 
described  in  more  detail. 

D  Chart.  This  is  an  attribute  chart 
(associated  with  discrete  values)  which  measures  the 
fraction  of  defective  parts  for  a  process  of  varying  size 
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Table  2.  Types  of  Control  Charts 


Character  is  tic _ Name 


Continuous  value  xR  chart  (avg  value  and  range) 

X  chart  (measured  value) 

Discrete  value  pn  chart  (number  of  defective 

units ) 

p  chart  (fraction  defective) 

c  chart  (number  of  defects) 

u  chart  (number  of  defects  per 

unit ) 


(8:74;  35:95;  31:77).  It  establishes  apper  and  lower 
control  limits  for  nonmeasurable  characteristics  of  a 
product  that  require  monitoring,  such  as  number  of 
typographical  errors,  mislabeled  items  and  defects  (8:75; 
55:115).  According  to  Juran  and  Gryna,  "the  inspection 
scheme  is  designed  to  serve  both  for  product 
acceptance  and  for  supplying  operator  control  data" 
(33:343).  Figure  7  displays  an  example  of  a  p  chart. 

xR  Chart.  This  is  an  average/range  chart. 
Lilce  the  p  chart  it  establishes  upper  and  lower  control 
limits  for  the  items  being  measured.  UnliJte  the  p  chart, 
the  xR  chart  is  used  to  evaluate  variable  items  such  as 
length,  temperature,  volume,  and  voltage  (continuous 
values )( 55 : 115;  35:95).  In  an  xR  chart  there  are  two 


54 


Figure  7.  p  Chart  Example 


distinct  areas.  The  x  portion  of  the  chart  "is  a  plot  of 
the  means  of  the  samples  taken  from  a  process"  (10:185). 
The  R  portion  of  the  chart  "is  a  plot  of  the  range  within 
each  sample"  (10:185).  The  R  chart  is  used  together  with 
the  X  chart  to  control  variation  within  subgroups  (35:95). 
Figure  8  displays  an  example  of  an  xR  chart. 

Summary  of  Quality  Control 

Quality  control  has  become  a  useful  and  increasingly 
necessary  management  requirement.  With  today's  increasing 
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Quality  control  has  become  a  useful  and  increasingly 
necessary  management  requirement.  With  today's  increasing 
costs,  decreasing  budgets,  and  customer  demand  for  high 
level  service,  the  question  concerning  quality  is  not  if, 
but  how.  In  order  to  implement  quality  control  effectively. 
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Figure  8.  Average/Range  Chart  Example 

managers  must  understand  the  basic  principles  upon  which  it 
is  founded.  Once  this  foundation  is  established,  management 
can  then  turn  to  identifying  the  measures  and  tools 
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necessary  to  implement  quality  products  and  to  make 
continuous  improvement  a  reality. 

Chapter  II  Summary 

In  this  chapter,  a  detailed  review  was  presented 
addressing  the  history  of  the  CIP,  the  automated 
enhancements  of  the  CIP,  and  a  background  on  quality  control 
techniques.  Through  this  discussion,  the  problems  affecting 
the  CIP  were  thoroughly  presented  along  with  the  actions 
necessary  to  initiate  the  improvement  process.  A  review  of 
the  literature  further  substantiates  that  the  measures  of 
percent  defects  and  cycle  time  could  apply  to  the  CIP. 
Furthermore,  the  literature  suggests  these  measures  could 
provide  an  opportunity  for  establishing  continuous  process 
improvement.  Based  on  these  conclusions,  a  methodology  was 
developed  to  investigate  the  potential  of  these  measures. 
Chapter  III  details  this  methodology  providing  the  reader 
with  a  step  by  step  plan  of  how  the  primary  research  in  this 
study  was  obtained.  The  findings  and  analysis  of  Chapter 
IV,  were  based  on  implementation  of  this  methodology,  and 
has  subsequently  established  the  conclusions  and 
recommendations  contained  in  Chapters  V.  Because  the 
methodology  and  resulting  conclusions  were  based  heavily  on 
the  literature  review,  the  reader  is  enco.  aged  to  be 
thoroughly  familiar  with  the  concepts  contained  in  Chapter 
II  before  reviewing  the  remainder  of  this  study. 
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III.  Methodolo 


Introduction 

This  chapter  outlines  the  process  and  procedures  used  in 
data  collection  and  analysis  for  the  purpose  of  answering 
the  investigative  questions  posed  in  Chapter  I.  Several  of 
these  areas  have  already  been  introduced  to  the  reader  in 
the  first  two  chapters  to  enhance  the  understanding  of  eaci. 
area's  importance  and  interaction  within  the  study.  Chapter 
III  expands  on  those  introductions  and  integrates  all  other 
pertinent  characteristics  or  peculiarities  surrounding  their 
implementation.  This  chapter  also  identifies  the 
limitations  of  this  research  based  on  measurement  techniques 
and  internal  and  external  validity  of  the  data. 


Specific  Problem  Revisited 

As  identified  in  Chapters  I  and  II,  numerous  problems 
currently  affect  the  performance  of  the  Air  Force  Critical 
Item  Program.  These  problems  include  inaccurate  critical 
item  selection  criteria,  insufficient  management  attention, 
and  reactionary  management  practices.  More  significant  than 
these  shortfalls  is  the  clear  lack  of  a  means  for  evaluating 
the  performance  of  the  overall  program.  Because  no 
evaluation  capability  of  the  critical  item  program  exists, 
the  quality  of  its  performance  and  product  is  consequently 
uncontrollable.  This  management  shortfall  results  in  th2 


specific  problem  statement  addressed  by  this  study.  The 
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problem  statement  identified  in  Chapter  I,  along  with  the 
applicable  investigative  and  measurement  questions,  are 
presented  again  to  refresh  the  reader's  memory. 

Problem  Statement.  While  the  Critical  Item  Program 
(CIP)  is  known  not  to  work  well,  no  specific  measures  of  the 
program  exist.  The  quality  literature  suggests  that  two 
measures,  percent  defects  and  cycle  time,  might  work; 
however,  no  one  has  evaluated  these  measures  with  respect  to 
managing  the  CIP. 

Investigative  and  Measurement  Questions.  To  answer  the 
question  posed  in  the  problem  statement,  two  investigative 
questions  were  addressed.  In  turn,  to  satisfy  each 
investigative  question,  several  mcti^urement  questions  are 
asked.  The  two  investigative  questions  and  their  associated 
measurement  questions  are  as  follows: 

Investigative  Question  #1.  Does  measurement  of 
percent  defects  aid  the  Air  Force  in  managing  the  CIP 
effectively? 

Associated  Measurement  Questions: 

1.  Do  items  in  the  CIP  meet  MAJCOM  criteria 

lAW  AFM  67-1? 

2.  Do  items  in  the  CIP  correspond  to  items 
identified  by  WSMIS/GWAM? 

3.  Do  items  enter  and  leave  the  CIP  on  a 
recurring  basis? 
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4.  How  does  this  measure  correlate  to 
aircraft  availability  in  terms  of  total  not  mission  capable 
(MICAP)  period  hours? 

5.  Do  CIP  managers  find  this  measure  useful? 

Investigative  Question  #2.  Does  measurement  of 

cycle  time  aid  the  Air  Force  in  managing  the  CIP 
effectively? 

Associated  Measurement  Questions; 

6.  What  is  the  variation  in  time  for  items 
managed  in  the  CIP? 

7.  How  does  this  measure  correlate  to 
aircraft  availability  in  terms  of  total  MICAP  period  hours? 

8.  Do  CIP  managers  find  this  measure  useful? 

Research  Design 

Before  detailing  the  specific  processes  and  procedures 
used  in  this  study  it  is  helpful  for  the  reader  to 
understand  the  research  design.  With  this  knowledge  one 
better  understands  why  certain  methodologies  are  chosen  over 
others.  This  discussion  also  enhances  comprehension  of  the 
study  limitations  based  on  the  methodologies  used.  Table  3 
summarizes  the  overall  research  design  of  this  thesis. 

The  problem  focus  of  this  study  is  formal.  Since  the 
Critical  Item  Program  (CIP)  has  existed  for  many  years,  the 
problems  associated  with  it  have  passed  exploratory  study 
and  can  now  be  described  specifically  and  concretely. 
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Table  3.  Research  Design  (18:58-61) 


PROBLEM  FOCUS . . 
DATA  COLLECTION 

VARIABLES . 

PURPOSE . 

TIME . 

SCOPE . 

ENVIRONMENT. .  .  . 


FORMAL 

OBSERVATION 
&  SURVEY 

EX  POST  FACTO 

DESCRIPTIVE 

CROSS-SECTIONAL 

CASE 

FIELD  STUDY 


Because  the  program  has  been  around  for  some  time^ 
limited  access  to  existing  data  was  available.  Along  with 
the  availability  of  an  existing  data  base,  there  were  also 
numerous  professionals  charged  with  manipulating  these  data 
to  manage  the  program.  This  study  uses  observational  and 
survey  data  collection  techniques  based  on  the  availability 
of  the  data  and  field  experts.  Data  are  collected  after  the 
fact,  defining  the  variables  as  ex  post  facto.  This  study 
attempts  to  show  relationships  among  variables  of  the 
critical  item  program  by  demonstrating  how  quality  control 
measures  can  be  useful  in  evaluating  program  performance. 
This  relationship  is  specific  in  nature  and  therefore 
classifies  as  a  descriptive  versus  causal  purpose. 

The  limited  availability  of  historical  data  prevents 
this  study  from  having  a  pure  statistical  scope.  Though 
statistical  techniques  were  used,  they  were  at  a  level  which 
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precluded  achieving  true  statistical  significance.  Because 
of  the  limited  availability  of  historical  data  and 
subsequent  statistical  application,  the  research  was 
appropriately  classified  as  a  case  study.  Finally,  as  this 
study  focuses  on  real  world  activities  and  actual  data,  the 
environment  of  the  study  specifically  addresses  a  field 
application. 


Primary  Source  Data  Collection 

As  discussed  earlier  in  this  chapter,  this  study  employs 
both  observation  and  survey  techniques  in  the  process  of 
primary  data  collection.  Together,  these  two  techniques 
provide  a  solid  cross  section  of  sources  in  validating  the 
measures  in  question  (18:183-184).  This  section  separately 
describes  specific  data  gathered  by  each  technique  due  to 


the  substantial  differences  in  data  collection  methods. 


Observation .  The  CIP  is  currently  undergoing  automated 
improvements.  These  ongoing  improvements  limit  the  size  of 
the  CIP's  population  which  currently  reflects  newly 
automated  changes.  Based  on  this  limitation,  SAC  is  the 
only  MAJCOM  observed  in  this  study.  Specific  weapon  systems 
observed  include  the  B-1,  B-52,  C-135,  F-111,  and  E-4 
aircraft.  A  100%  analysis  of  available  data  was 
accomplished  for  each  of  these  aircraft  systems  in  the 
observation  phase.  This  100%  analysis  limits  the  bias  and 
error  of  the  study's  internal  validity  (18:277).  The 
primary  source  of  data  comes  from  information  contained  in 
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WSMIS/GWAM.  The  reports  used  from  WSMIS/GWAM  include  the 
approved  critical  item  list,  the  critical  item  add/delete 
report,  the  critical  item  candidate  list,  and  the  automated 
Form  74.  Observation  of  data  occurs  in  a  monthly  and/or 
quarterly  cycle  depending  on  availability  of  reports. 

Through  analysis  of  data  observed  and  application  of  the  two 
measures  in  question,  these  data  were  used  to  establish  the 
current  condition  of  the  CIP  for  the  SAC  aircraft  stated 
above . 

Of  special  note  was  the  collection  of  cycle  time  data. 

In  order  to  determine  true  cycle  time  measures,  current  as 
well  as  completed  items  should  be  gathered  to  determine  this 
measure.  However,  this  was  not  possible  for  this  study  as 
no  capability  exists  to  aggregately  gather  cycle  time  data 
on  items  currently  maintained  in  the  program.  The  only 
ability  to  measure  cycle  time  data  was  for  items  leaving  the 
CIP  through  analysis  of  Add/Delete  reports. 

Archived  Data.  This  study  establishes  a  baseline 
thr..-ugh  observation  of  archived  data  gathered  from  other 
sources  such  as  Air  Logistic  Center  (ALC)  Critical  Item 
Man  igers  (CIM),  System  Program  Managers  (SPM),  Major  Command 
(MAJCOM)  focal  points  and  automated  Forms  74.  The  format 
and  report  contents  of  this  archived  information,  with  the 
exception  of  automated  Forms  74,  were  not  comprehensive  nor 
100%  consistent  with  measurement  question  requirements  and 
therefore  prevented  the  use  of  statistical  significance 
testing.  These  data,  however,  do  provide  an  acceptable 
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baseline  for  comparison  to  data  collected  under  controlled 
conditions . 

Specifically,  gathering  of  archived  data  for  presence  of 
percent  defects  and  cycle  time  provided  a  baseline 
comparison  for  measurement  questions  2,  3,  4,  6,  and  7.  For 
all  SAC  CIP  items  available  through  archived  data,  a 
comparison  of  the  approved  critical  item  list  and  WSMIS/GWAM 
critical  item  candidate  list  (or  alternate  source 
equivalent)  was  performed.  As  historical  information  (such 
as  GWAM  Candidate  Lists)  was  not  100%  available,  some  data 
were  missing.  These  lists  and  their  equivalents  describe 
the  items  in  the  approved  CIP  and  items  nominated  as 
candidates  to  the  CIP.  Included  in  these  reports  were  the 
item  specific  criteria  that  were  attained  for  candidate 
items  to  become  nominated  by  WSMIS/GWAM.  This  comparison 
provides  a  baseline  for  measurement  question  2.  An  analysis 
of  archived  automated  Forms  74  provides  information  on  how 
long  items  remain  in  the  CIP  and  the  historical  monthly 
status  of  these  items  prior  to  their  release  from  the 
program.  This  review  provided  a  baseline  for  measurement 
question  6. 

Upon  collection  of  these  data,  separate  comparisons  were 
accomplished  between  trends  in  percent  defects  and  cycle 
time  to  aircraft  availability.  For  the  purposes  of  this 
study,  aircraft  availability  was  based  on  the  combination  of 
not  mission  capable  supply  (NMCS)  and  not  mission  capable 
both  (NMCB)  data  (45).  Together,  NMCS  and  NMCB  establish 
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nonavailability  o£  aircraft  due  to  supply  actions.  Because 
the  CIP  primarily  effects  supply  and  not  maintenance,  not 
mission  capable  due  to  maintenance  (NMCM)  is  not  considered 
in  the  comparison.  NMCS  and  NMCB  data  are  collected  from 
the  WSMIS/Readiness  Assessment  Module  (RAM)  using  total 
MICAP  period  hours.  Comparison  of  these  trends  identified 
whether  a  nonparametr ic  correlation  existed  between  the 
performance  of  the  critical  item  program  and  the  operational 
availability  of  aircraft.  These  data  further  provided  a 
baseline  for  measurement  questions  4  and  7. 

Archived  data  do  not  exist  to  identify  specific  criteria 
attained  for  approved  critical  items  not  identified  by 
WSMIS/GWAM.  This  condition  results  from  MAJCOMs  using 
unique  criteria  or  judgment  to  s.elect  critical  items. 
Currently,  these  unique  critical  item  selections  do  not 
require  justification  or  any  form  of  reporting  (15). 
Therefore,  no  historical  data  were  available  to  provide  a 
baseline  for  measurement  question  1. 

Current  Data.  Unli)ce  archived  data,  current  data 
(data  collected  beginning  1  April  1991)  were  collected  in  a 
format  specifically  designed  to  address  applicable 
measurement  questions.  As  consistency  and  format  of  current 
data  collection  were  completely  controlled,  this  study 
places  more  reliability  and  weight  on  these  data. 

Observation  of  current  data  solely  uses  WSMIS/GWAM 
reports.  Subsets  of  data  for  MAJCOM  and  weapon  system  were 
created  in  WSMIS/GWAM  for  generation  of  approved  critical 
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item  lists,  critical  item  add/delete  reports,  critical  item 
candidate  lists,  and  automated  Forms  74.  Analysis  of  these 
data,  like  that  achieved  through  archived  data,  provided 
inputs  to  answering  measurement  questions  2,  3,  4,  6,  and  7. 
Through  contacts  made  possible  by  the  WSMIS/GWAM  Program 
Manager,  additional  data  were  to  be  collected  which  detailed 
specific  criteria  attained  for  unique  items  selected  by  the 
MAJCOM  (items  not  identified  by  WSMIS/GWAM  as  critical). 
These  additional  data  were  to  provide  the  basis  for 
answering  measurement  question  1. 

This  study  again  analyzes  current  data  to  NMCS  and  NMCB 
to  determine  if  a  nonparametr ic  correlation  existed  between 
CIP  performance  and  aircraft  availability.  Together  with 
archived  data,  measurement  questions  4  and  7  were  answered. 

Survey.  The  last  method  of  primary  source  data 
collection  was  survey.  A  presentation  of  the  above 
observations  and  conclusions  were  provided  to  critical  item 
managers  from  all  levels  in  the  program  (see  Appendix  C). 
Managers  addressed  include  ALC  CIMs,  SPMs,  SAC  focal  points, 
and  Headquarters  AFLC  critical  item  focal  points.  Twenty 
three  managers  were  addressed  in  the  survey  process  (see 
Table  4).  Of  these  twenty  three  managers,  twenty  two 
responded.  The  presentation  of  findings  provided  complete 
analysis  and  conclusions  determined  through  observed  data 
which  addressed  measurement  questions  1,  2,  3,  and  6.  Also 
included  in  the  presentation  was  a  complete  definition  and 
description  of  how  percent  defects  and  cycle  time  were  used 
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Table  4.  Critical  Item  Experts  Interviewed 


Position 


Number  Interviewed 


ALC  Division  Chiefs  5 
SPM  5 
MAJCOM  POC  4 
CIM  4 
HQ  APLC  2 
HQ  USAF  1 
AFAA  1 


in  this  analysis .  After  presentation  of  the  analysis, 
managers  were  asked  (via  telephone  interview)  to  respond  to 
several  questions  (see  Appendix  C).  The  questions  were 
designed  to  be  general  and  open  ended  to  enable  managers  to 
respond  freely  and  without  bias.  This  questionnaire  asked 
their  opinion  addressing  the  utility  of  percent  defects  and 
cycle  time  in  evaluating  the  performance  of  the  critical 
item  program.  Managers  were  asked  to  briefly  support  their 
answers.  Due  to  the  nature  of  the  questions  posed, 
validation  of  the  interview  instrument  was  difficult.  To 
address  this  properly,  a  detailed  review  of  the  questions 
was  accomplished  through  several  phases.  First,  face 
validity  of  the  instrument  was  obtained  through  review  by  a 
faculty  advisor.  Second,  a  review  of  the  questions  were 
accomplished  through  preliminary  interview  with  local 
critical  item  management  experts.  From  their  comments,  the 
questionnaire  was  modified  for  final  application  to  the 
remaining  field  experts.  Through  this  process  ambiguous 
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questions  and  bias  were  eliminated  to  the  best  extent 
possible  given  the  limitations  of  the  environment. 

Remaining  experts  were  interviewed  using  the  updated 
questionnaire  upon  completion  of  the  validation  effort. 
Conclusions  for  measurement  question  5  and  8  were  determined 
from  this  survey  using  content  analysis  (34:525).  Through 
the  methods  of  observation  and  survey  all  seven  measurement 
questions  were  addressed.  Once  again,  this  combination  of 
primary  source  data  collection  techniques  was  expected  to 
provide  a  solid  cross  section  of  data  and  validity  to  the 
conclusions  of  this  study. 

Data  Analysis 

Having  described  how  primary  data  were  collected,  this 
section  describes  the  measurement  tools  employed  to  conduct 
the  analysis.  This  section  presents  the  statistical  quality 
control  (SQC)  charts  used,  the  mathematical  manipulations 
required,  and  the  limitations  associated  with  each. 

SQC  Charts.  Based  on  analysis  of  the  literature 
(Chapter  II)  the  SQC  charts  chosen  that  best  suit 
measurement  of  percent  defects  and  cycle  time  were  the  p 
chart,  and  the  average  and  range  chart  respectively.  Both 
of  these  charts  are  applicable  at  both  the  MAJCOM  and  Air 
Force  levels  for  measurement  of  the  CIP's  performance. 

p  Chart.  The  p  chart  was  chosen  to  measure  percent 
defects  over  other  control  charts  discussed  in  the 
literature  due  to  its  direct  application  to  attribute 
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measurement  (9:54;  27:210;  33:337;  35:63).  An  example  of  a 
p  chart  is  presented  in  Figure  9.  To  construct  this  chart, 
the  following  calculations  were  performed  (9:54;  31:77-79; 
33:334;  35:94): 

number  of  defects 
number  inspected 

—  total  number  of  defects  ^2) 

total  number  inspected 


Upper  Control  Limit 


UCL^ 


■p+aJ 


p(i-p) 


(3) 


where : 

UCL  =  upper  control  limit 
Pfe>.>K  =  average  number  of  defects 
n  =  sample  size 


Lower  Control  Limit 


LCL^p~3^ 


P(l-P) 

n 


(4) 


where : 

LCL  =  lower  control  limit 
Pbav  =  average  number  of  defects 
n  =  sample  size 


The  advantages  associated  with  using  this  chart  are  that  it 
is  easily  understood,  it  provides  an  overall  picture  of 
quality,  and  the  data  are  usually  readily  available 
(33:337).  The  disadvantages  of  this  chart  include  that  it 
does  not  provide  detailed  information  of  individual 
characteristics  and  it  does  not  recognize  differing  levels 
of  defectiveness  (33:337). 

Average  and  Range  (xR)  Chart.  The  xR  chart  was 
chosen  to  measure  cycle  time  due  to  its  direct  application 
to  measurement  of  continuous  values  ( variables )( 9 : 53; 
27:211;  31:65;  33:337;  35:95).  An  example  of 
an  xR  chart  is  provided  in  Figure  10.  The  xR  chart 
provides  control  limits  which  aid  managers  in  determining 
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whether  data  are  in  or  out  of  control.  This  ability  to 
recognize  outliers  enables  managers  to  react  quickly  and 
adjust  the  process  as  required.  Mathematical  calculations 
required  to  construct  and  administer  an  xR  chart  are 
provided  below: 


Sample  Average 


(5) 


where : 


X  13 


critical  item  cycle  time 
number  of  samples  taken 
average  cycle  time 


X 

k 


Sample  Range 


(6) 


where : 

R  =  range  over  which  sample  data  are  spread 
k  =  number  of  samples  taken 
Rto»  =  average  range 


Special  factors  have  been  developed  that  are  required  to 
calculate  upper  and  lower  control  limits.  These  factors  are 
listed  in  Table  5  below  (8:195). 


Average  Upper  Control  Limit 


(7) 


where : 

UCLx  =  upper  control  limit  for  X  bar 

=  average  cycle  time  of  weapon  systems  or 
MAJCOMs  sampled 

Rt3«»  =  average  range  over  which  sample  data  are 
spread 

Aa  =  special  factor  for  control  limit 


Average  Lower  Control  Limit 

LCL^Tt~Ajt 


(8) 


where : 


LCLx 


Rbae 


Aa 


lower  control  limit 
average  cycle  time 
MAJCOMs  sampled 
average  range  over 
spread 

special  factor  for 


for  X  bar 

of  weapon  systems  or 
which  sample  data  are 
lower  control  limit 
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Range  Upper  Control  Limit 


(9) 


UCLjf^D^'R 


where  : 

UCLr  =  upper  control  limit  for  range 
Rto.s^  =  average  range  over  which  sample  data  are 
spread 

D4  =  special  factor  for  upper  control  limit 


Table  5.  Concrol  Factor  Limits 


n  Aa  D4 


2 

1 . 880 

3 .268 

3 

1 . 023 

2.574 

4 

0 .729 

2 . 282 

5 

0.577 

2 .114 

6 

0 . 483 

2 . 004 

n 

=  number 

in  the  sample 

A2 

=  special 

factor  for 

lower 

control 

limit 

D4 

=  special 

factor  for 

upper 

control 

limit 

Once  the  data  are  collected,  the  sample  average  and 
range  (equations  5  &  6)  are  plotted  on  the  upper  and  lower 
grid  of  the  control  chart.  Normally  30  points  of  data  are 
required  to  establish  meaningful  upper  and  lower  control 
limits  (10:181).  Availability  of  sufficient  historical 
longitudinal  data  falls  short  of  this  requirement  as  only 
three  periods  (quarters)  worth  of  data  were  present. 
Because  of  this  shortfall,  the  upper  and  lower  control 
limits  must  be  considered  starting  points  which  are  to  be 
recalculated  and  adjusted  as  more  data  become  available  in 
the  future  and  the  program  progresses. 
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In  short,  these  control  limits  act  as  bounds  to 
determine  whether  data  are  in  or  out  of  control.  As  more 
control  is  exercised  over  the  program,  these  limits  contract 
signifying  less  variation  in  the  process.  Therefore, 
management's  goal  should  be  to  get  the  system  into  control 
and  then  to  shrink  the  control  limits  to  as  small  as 
practical . 

Scatter  Diagram.  In  addition  to  the  p  and  xR 
control  charts,  the  scatter  diagram  was  used  to  display  the 
possible  existence  of  correlation  between  percent  defects 
and  cycle  time  to  aircraft  availability.  To  determine  if  a 
correlation  existed  between  the  two  measures  and  aircraft 
availability,  two  models  were  developed.  The  first  model 
compared  percent  defects  to  aircraft  availability  and  the 
second  model  compared  cycle  time  to  aircraft  availability. 
Spearman's  Rank  Correlation  of  Coefficient  was  the 
calculation  chosen  to  determine  correlation  (38:979-983). 
This  calculation  was  used  due  to  the  non-normal  distribution 
of  available  data,  thereby  requiring  a  non  parametric 
technique  be  used  (30).  Mathematical  calculations  required 
to  complete  Spearman's  calculation  are  as  follows: 

Sum  of  Squares  uv 

where : 

SSuv  =  sum  of  squares  uv 

Ui  =  rank  of  the  ith  measurement  in  sample  1 

Vi  =  rank  of  the  ith  measurement  in  sample  2 

Ub«E  =  average  rank  of  sample  1 

Vb«*  =  average  rank  of  sample  2 
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Sum  o£  Squares  uu 


(11) 


where : 

SSuu  =  sum  of  squares  uu 

Ui  =  rank  of  the  ith  measurement  in  sample  1 
Uto.ir  =  average  rank  of  sample  1 


Sum  of  Squares  vv 


(12) 


where : 

SSw  =  sum  of  squares  uu 

Vi  =  rank  of  the  ith  measurement  in  sample  2 
=  average  rank  of  sample  2 


Spearman's  Rank  Correlation  Coefficient 


r 


(13) 


where : 

Zm  =  Spearman's  rank  correlation  of  coefficient 
SSov  =  sum  of  squares  uv 
SSuu  =  sum  of  squares  uu 
SSvv  =  sum  of  squares  vv 


Correlation  of  the  two  measures  to  aircraft  availability 
is  based  on  the  value  of  r.  As  r  approaches  one  or  negative 
one  there  is  evidence  that  a  correlation  exists  (38:981- 
982).  If  r  remains  close  to  zero  there  is  evidence  that  no 
correlation  exists  (38:982).  As  total  MICAP  period  hours 
were  used  as  the  measure  of  aircraft  availability  (a 
reflection  of  aircraft  non-availability),  a  positive  r  value 
was  desired  to  establish  the  existence  of  a  positive 
correlation  with  the  measures  percent  defects  and  cycle 
time.  The  measurement  of  correlation  is  displayed  using  a 


75 


scatter  diagram  plot  with  a  significance  level  of  .05  (p  < 
.05)  (see  Figure  11).  The  software  package  of  Statistix  3.1 
was  used  to  calculate  Spearman's  Rank  Correlation  of 
Coefficient  (51:95). 

Survey  Measurement.  Data  collected  from  the  survey  of 
critical  item  managers  were  analyzed  using  content  analysis 
(34:525).  Each  manager  interviewed  was  categorized  into  a 
specific  job  function,  for  example,  ALC  CIM,  SPM,  MAJCOM 
focal  point,  etc.  In  addition,  each  manager's  response 
concerning  percent  defects  and  cycle  time  was  categorized  as 
either  recommended,  not  recommended,  or  no  opinion  (see 
Table  6).  Categorization  of  manager  responses  in  this 
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manner  provided  nominal  measurement  of  data  and  identified 
existing  trends  among  the  various  positions.  To  determine 
these  trends,  interviews  were  transcripted  and  in  turn 
content  analysis  performed.  A  general  and  occupational 


Table  6.  Management  View  of  Percent  Defects/Cycle  Time 


Position  %  Defects  Cycle  Time  It  Int 

No  No  No  No 

Rec  Rec  Opn  Rec  Rec  Opn 


Div  Chf 
SPM 

MAJCOM 

CIM 

HQ  AFLC 
HQ  LEYS 
AFAA 


Totals : 


consensus  of  opinions  was  compiled  to  display  results  of  the 
survey.  The  experts  interviewed  in  this  study  represented 
approximately  the  entire  population  for  the  CIP  in  relation 
to  the  weapon  systems  analyzed.  Due  to  this  fact,  no 
statistical  test  was  considered  necessary  and  would  have 
been  inapprppr iate  to  determine  the  significance  level  of 
the  results  ( 30 ) . 

Chapter  III  Summary 

This  chapter  has  outlined  the  process  and  procedures 
used  in  data  collection,  data  analyses,  mathematical 
computation  and  survey  organization.  Basically  two  data 
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collection  methods  were  used,  observation  and  survey.  The 
major  source  of  observed  data  came  from  the  WSMIS/GWAM 
computer  information  system.  Experienced  critical  item 
managers  in  the  field  were  presented  with  observational 
conclusions  and  then  surveyed  for  their  opinion  of  the 
utility  of  percent  defects  and  cycle  time.  The  SQC  charts 
chosen  for  use  in  this  study  were  the  p  and  xR  charts.  Also 
used  was  the  scatter  diagram  to  show  the  existence  of 
correlation.  Spearman's  Rank  Correlation  Coefficient  was 
used  to  determine  if  correlation  existed  between  the  two 
measures  of  this  study  and  aircraft  availability. 
Implementation  of  these  concepts  and  the  resulting 
conclusions  are  presented  in  Chapter  IV. 
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IV.  Findings  and  Analysis 


Introduction 

This  chapter  presents  the  findings  resulting  from 
applying  the  methodology  presented  in  Chapter  III.  Due  to 
non-availability  of  data,  minor  deviations  from  that  plan 
were  necessary.  These  deviations  are  fully  outlined  before 
presenting  the  research  findings  and  analysis. 

Specifically,  this  chapter  presents  the  developed  control 
charts  used  to  demonstrate  how  to  evaluate  the  performance 
of  the  Critical  Item  Program  (CIP)  using  the  measures  of 
percent  defects  and  cycle  time,  the  calculations  of 
Spearman's  Rank  Correlation  of  Coefficient  between  the  two 
measures  and  aircraft  availability,  and  the  opinions  of 
critical  item  management  experts  concerning  the  use  of  these 
measures  in  the  field. 

Limitations  of  the  Data 

As  mentioned  above,  the  non-availability  of  data  made 
it  necessary  to  deviate  from  the  established  methodology. 
These  deviations,  although  minor  in  nature,  do  effect  the 
level  of  confidence  in  the  overall  data  presented.  To 
ensure  the  reader  is  aware  of  these  shortfalls,  these 
limitations  are  described  below. 

For  the  quarters  reviewed  (ending  in  the  months  of  June 
1990,  September  1990,  January  1991,  and  April  1991),  the 
Weapon  System  Management  Information  System  (WSMIS)/Get  Well 
Assessment  Module  (GWAM)  Critical  Item  Candidate  List  was 


79 


not  available  in  its  entirety  for  Strategic  Air  Conunand 
(SAC)  aircraft  for  the  quarters  ending  September  1990  and 
January  1991.  The  only  GWAM  Critical  Item  Candidate  List 
data  that  were  available  for  these  two  quarters  were  for 
items  managed  by  Warner-Robins  Air  Logistic  Center  (ALC). 

To  provide  measurable  data  points  for  the  September  1990  and 
January  1991  quarters,  Warner-Robins  ALC  managed  items  were 
reviewed  as  a  subset  of  the  SAC  CIP.  Therefore,  data 
presented  for  percent  defects  for  September  1990  and  January 
1991  reflect  items  managed  at  Warner-Robins  ALC  only  and  not 
all  SAC  aircraft.  Though  this  limits  the  confidence  of  the 
measures  for  these  two  quarters  in  respect  to  the  overall 
SAC  program,  the  use  of  the  data  does  demonstrate  how  these 
measures  can  be  applied  by  different  levels  and 
organizations  of  the  Air  Force  using  the  same  techniques. 

Another  shortfall  was  the  inability  to  determine 
whether  items  currently  in  the  CIP,  that  were  not  candidates 
recommended  by  WSMIS/GWAM,  met  the  criteria  specified  in  Air 
Force  Manual  (AFM)  67-1.  Individuals  previously  contacted 
who  were  to  provide  these  data  were  unable  to  do  so  due  to 
other  pending  requirements.  Because  of  these  missing  data, 
it  was  not  possible  to  quantitatively  answer  measurement 
question  1. 


Findings 


The  findings  in  this  chapter  include  the  data  collected 
to  demonstrate  the  measures  of  percent  defects  and  cycle 


80 


time,  the  correlation  of  these  measures  to  aircraft 
availability  (using  MICAP  period  hours),  and  the  potential 
of  these  measures  in  field  use  according  to  experienced 
critical  item  managers. 

Percent  Defects.  Data  used  to  determine  the  measure  of 
percent  defects  for  each  of  the  five  SAC  aircraft  contained 
in  this  study  were  collected  from  SAC  Critical  Item  lists, 
WSMIS/GWAM  Critical  Item  Candidate  lists,  and  WSMIS 
Automated  Forms  74.  Results  of  these  efforts  are  provided 
in  Table  7.  From  these  data,  p  charts  were  developed  to 
demonstrate  the  use  of  percent  defects  in  evaluation  of  the 
CIP.  As  a  reminder,  defects  include  items  in  the  CIP  which 
were  not  recommended  by  WSMIS/GWAM,  items  that  enter  and 
leave  the  CIP  on  a  recurring  basis,  and  items  in  the  CIP 
that  do  not  meet  preestablished  MAJCOM  criteria. 

Due  to  insufficient  historical  information  (generally 
20  -  30  historic  data  points),  meaningful  upper  control 
limits  (UCL)  were  not  able  to  be  calculated  using  the 
formulas  identified  in  Chapter  III.  In  light  of  this 
shortfall,  a  UCL  of  30%  was  implemented  based  on  the 
expected  CIP  performance  according  to  critical  item 
management  officials  (36). 

With  the  UCL  set  at  30%,  the  p  control  charts  (Figures 
12  -  17)  clearly  illustrate  that  most  measures  for  percent 
defects  exceed  the  UCL. 

The  E-4  percent  defects  measures  (Figure  12)  were  very 
high,  equalling  or  exceeding  100%  for  two  of  the  four 
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Table  7.  Percent  Defects  Data 


B-52  Jun/90  Sep/90  Jan/91  Apr/91 


Critical  items 

in  CIP: 

62 

23 

30 

86 

Critical  items 

not 

r econuiiended  by 

GWAM: 

32 

15 

13 

28 

Recurring  items 

: 

n/m 

n/m 

n/m 

34 

CIP  items  not  meeting 
MAJCOM  criteria  lAW 


AFM  67-1: 

n/m 

n/m 

n/m 

n/m 

C-135 

Critical  items  in  CIP: 

100 

22 

18 

115 

Critical  items  not 
recommended  by  GWAM: 

62 

10 

6 

53 

Recurring  items: 

n/m 

n/m 

n/m 

30 

CIP  items  not  meeting 
MAJCOM  criteria  I AW 

AFM  67-1: 

n/m 

n/m 

n/m 

n/m 

B-1 

Critical  items  in  CIP: 

33 

8 

8 

22 

Critical  items  not 
recommended  by  GWAM: 

3 

4 

3 

7 

Recurring  items: 

n/m 

n/m 

n/m 

7 

CIP  items  not  meeting 
MAJCOM  criteria  I AW 

AFM  67-1: 

n/m 

n/m 

n/m 

n/m 

E-4 

Critical  items  in  CIP: 

6 

5 

5 

5 

Critical  items  not 
recommended  by  GWAM: 

5 

4 

5 

4 

Recurring  items: 

n/m 

n/m 

n/m 

2 

CIP  items  not  meeting 
MAJCOM  criteria  I AW 

AFM  67-1: 

n/m 

n/m 

n/m 

n/m 
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Table  7  (continued). 


F-111 

Jun/90 

Sep/90 

Jan/91  Apr/91 

Critical  items  in  CIP: 

20 

2 

0 

0 

Critical  items  not 
recommended  by  GWAM: 

3 

0 

0 

0 

Recurring  items: 

n/m 

n/m 

n/m 

n/m 

CIP  items  not  meeting 
MAJCOM  criteria  I AW 

AFM  67-1: 

n/m 

n/m 

n/m 

n/m 

NOTE:  n/m  indicates  the 
measured . 

data  were  not 

able  to 

be 

periods.  This  particular  weapon  system  has  very  few  items 
in  the  CIP  due  to  its  contractor  logistics  support  (CLS) 
resupply  structure.  As  a  result,  if  only  a  few  items  have 
defects  of  those  items  in  the  CIP,  high  percent  defects 
measurements  will  result. 

The  F-111  (Figure  13)  is  another  weapon  system  with 
very  little  activity  in  the  CIP.  Because  the  F-111  items 
that  were  in  the  CIP  during  the  first  quarter  contained  no 
detect  conditions,  a  low  percent  defects  measure  resulted. 

In  the  final  three  quarters  for  the  F-111  aircraft,  no  items 
were  added  to  the  CIP  resulting  in  zero  defects  due  to  zero 
items . 

The  C-135  is  one  of  SAC's  more  active  weapon  systems  in 
the  CIP.  Tnis  is  evident  by  the  number  of  items  entered 
into  the  CIP  each  quarter  (see  Figure  14).  The  C-135  p 
control  chart  demonstrates  a  fluctuating  performance  with 
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84 


MAJCOM;  SAC 
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percent  defects  measures  ranging  from  33  to  72  percent. 

This  fluctuation  is  consistent  with  control  charts  showing 
out  of  control  processes,  which  suggests  that  the  CIP  is 
currently  not  in  control. 

The  B-1  is  also  an  active  performer  in  the  CIP.  Like 
the  C-135,  the  B-l's  performance  (Figure  15)  also  fluctuated 
widely,  ranging  from  9  to  63  percent.  Though  the  B-l's 
initial  guarter's  performance  fell  below  the  UCL,  the 
remaining  quarters  all  registered  measures  above  the  process 
control  range.  This  control  chart,  identifies  wide 
variability  and  suggests  a  lack  of  control  being  experienced 
in  the  CIP. 

Figure  16  presents  the  performance  of  the  B-52.  Like 
the  C-135  and  B-1,  the  B-52  displayed  fluctuating 
performance  above  the  established  UCL.  This  is  also 
indication  of  a  process  out  of  control. 

For  the  five  SAC  aircraft  overall,  the  average  percent 
defects  measure  was  50%.  This  high  percent  defect  rate 
indicates  the  strong  potential  for  improvement.  As  two  of 
the  three  defect  conditions  were  not  measured  for  the  first 
three  periods  of  the  study  (number  not  meeting  criteria  and 
number  of  recurring  items),  and  one  of  the  three  defect 
conditions  was  not  measured  in  the  fourth  period  of  the 
study  (both  shortfalls  were  due  to  non-availability  of  data 
and  process  time  requirements),  an  even  higher  percent 
defects  measure  would  probably  have  resulted  had  these 
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Figure  15.  B-1  p  Control  Chart 
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Figure  16.  B-52  p  Control  Chart 
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Figure  17.  SAC  Overall  p  Control  Chart 


90 


conditions  been  able  to  be  evaluated.  Consequently,  the 
large  variability  which  is  shown  in  these  control  charts 
would  likely  have  been  worse  had  the  other  data  been 
available. 

Correlation  of  Percent  Defects  to  Aircraft  Avail¬ 
ability.  Spearman's  Rank  Correlation  of  Coefficient  was 
used  to  determine  whether  a  correlation  existed  between  the 
measures  of  percent  defects  and  aircraft  availability.  The 
WSMIS/Readiness  Assessment  Module  (RAM)  was  accessed  to 
extract  MICAP  period  hours  for  each  quarter  corresponding  to 
each  of  the  five  mission  design  series  (MDS)(see  Table  8). 
Using  the  software  package  Statistix  3.1  (51),  Spearman's 

Table  8.  Aircraft  Availability  Data  to  be  Correlated 
to  Percent  Defects 


MICAP  Period  Hours 

Jun/90 

Sep/90  Jan/91 

Apr/91 

E-4 

8715 

9145  11005 

22563 

F-111 

0 

0  0 

0 

B-52 

594880 

1209852  1337557 

530672 

B-1 

616680 

609986  337790 

303221 

C-135 

1921 

3852  7598 

2977 

SAC  Overall 

1222196 

1832835  1693950 

859433 

NOTE:  MICAP  period  hours  are  the  total  hours  a 
weapon  system  is  not  mission  capable  due  to  the  lack 
of  specific  piece  of  equipment  or  spare  part. 


Rank  Correlation  of  Coefficient  was  calculated  comparing  the 
aircraft  availability  values  and  the  percent  defects  values 
for  each  MDS.  Results  of  these  calculations  are  provided  in 
Figures  18  -  22. 

As  identified  in  the  scatter  diagrams,  correlation 
values  for  percent  defects  to  aircraft  availability  ranged 
from  positive  .80  to  negative  .80.  This  range  by  itself 
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Figure  18.  E-4  Correlation  of  MICAP  Period 

Hours  to  Percent  Defects 
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Figure  19.  C-135  Correlation  of  MICAP  Period 


Hours  to  Percent  Defects 


Figure  20.  B-1  Correlation  of  MICAP  Period 

Hours  to  Percent  Defects 
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does  not  indicate  a  strong  or  consistent  correlation  between 
the  measure  of  percent  defects  and  aircraft  avai labi 1 i ty . 

In  order  to  establish  that  a  correlation  exists  within  a  95% 
confidence  level  requires  that  scores  be  greater  than  .90 
(38:1212).  As  none  of  the  scores  fulfilled  this 
requirement,  one  must  logically  conclude  that  a  correlation 
does  not  exist. 

In  addition  to  individual  measurement,  an  aggregate 
measure  of  percent  defects  to  MICAP  period  hours  was  also 
computed.  This  resulted  in  a  positive  correlation  of  .3033. 
In  order  to  reject  at  a  .05  level  of  significance  that  a 
correlation  did  not  exist,  a  value  of  greater  than  .377  was 
required  (38:1212).  As  this  was  not  achieved,  further 
support  was  established  that  there  was  no  correlation 
between  percent  defects  and  MICAP  period  hours. 

Because  the  ability  to  observe  each  defect  condition 
varied  among  the  four  periods  measured,  inflated  scores 
resulted  in  the  last  period  which  distorted  the  correlation 
outcomes.  To  attempt  to  compensate  for  this  situation, 
correlation  values  were  recalculated  comparing  percent 
defects  to  aircraft  availability  in  aggregate  using  a 
percent  defect  value  based  only  on  defect  condition  1  (those 
items  in  the  CIP  which  were  not  identified  by  WSMIS/GWAM), 
as  this  was  measured  consistently  throughout  all  four 
periods.  The  results  of  this  effort  provided  a  positive 
correlation  value  of  .3441.  Again,  in  order  to  reject  at 
the  .05  significance  level  that  no  correlation  existed,  a 


95 


value  greater  than  .377  was  required  (38:1212).  As  this 
value  was  not  achieved,  one  must  again  conclude  that  no 
correlation  exists  between  percent  defects  and  aircraft 
availability  at  the  .05  level  of  significance. 

One  final  correlation  was  run  between  the  total  number 
of  items  in  the  CIP  for  each  weapon  system  and  the 
respective  MICAP  period  hours  for  that  quarter.  Results  of 
this  calculation  provided  a  positive  correlation  of  .2031. 
However,  a  value  greater  than  .49  was  required  to  reject  at 
the  .05  level  of  significance  that  no  correlation  existed 
(38:1212) . 

The  confidence  in  the  findings  for  aircraft 
availability  and  percent  defects  is  limited  due  to  the  use 
of  partial  percent  defect  data  for  two  of  tiie  four  periods 
analyzed.  Confidence  in  correlation  results  would  be  much 
stronger  had  full  percent  defect  data  been  available  for  all 
four  periods  observed. 

Cycle  Time.  Data  used  to  determine  the  measure  of 
cycle  time  were  extracted  from  WSMIS/GWAM  Critical  Item 
Add/Delete  reports.  As  mentioned  in  Chapter  III,  this 
method  of  determining  cycle  time  only  measures  items  that 
have  completed  the  program.  This  was  a  shortfall  in  that  it 
did  not  consider  the  cycle  time  of  items  currently  in  the 
program;  however,  this  study  was  forced  to  accept  this 
limitation  as  no  other  means  of  cycle  time  measurement  was 
aval lable . 
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Data  were  collected  for  the  months  of  May,  June  and 
July  of  1991  for  the  five  SAC  aircraft  targeted  in  this 
study.  All  items  deleted  from  the  program  for  each 
respective  MDS  and  month  were  totaled  and  an  average 
calculated.  The  data  for  each  MDS  within  each  respective 
month  was  then  summed,  averaged,  and  ranged.  Only  three 
weapon  systems,  the  B-1,  C-135,  and  B-52,  had  measurable 
data.  The  other  two  weapon  systems  did  not  register  any 
delete  actions  in  the  three  months  observed,  resulting  in  no 
ability  to  measure  cycle  time  for  these  aircraft.  The  cycle 
time  calculations  for  each  of  the  observed  aircraft  and  SAC 
overall  are  provided  in  Figures  23  -  26. 

Once  again,  20  -  30  historical  data  points  were 
necessary  to  calculate  a  meaningful  UCL.  The  non¬ 
availability  of  historical  data  required  the  UCL  for  cycle 
time  to  be  established  based  on  critical  item  officials 
expectations  of  the  program  (36).  Review  of  the  charts 
identifies  that  all  three  aircraft  had  ranges  in  item  cycle 
time  that  fall  above  the  expected  UCLs .  The  B-1  had  a 
consistent  range  of  16  months,  the  C-135  ranged  were  from  7 
to  18  months,  and  the  B-52  had  a  consistent  range  of  18 
months.  As  both  UCLs  (average  and  range)  must  not  be 
exceeded,  one  must  conclude  that  all  three  aircraft  need 
management  attention  to  correct  cycle  time  deficiencies. 

As  indicated  by  the  SAC  overall  average/range  ( xR ) 
control  chart  (Figure  26),  the  average  cycle  time  for  SAC 
items  in  the  CIP  was  between  6.1  and  6.4  months.  In 
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Figure  25.  B-52  xR  Control  Chart 
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Figure  26.  SAC  Overall  xR  Control  Chart 
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addition,  cycle  time  for  SAC  items  in  the  CIP  had  an  average 
range  of  6.3  to  6.4  months.  The  average  and  range  values 
for  cycle  time  consistently  fell  below  the  hypothetical 
UCLs,  indicating  that  both  the  UCLs  could  be  lowered. 

Comparing  the  results  from  the  individual  MDS  xR  charts 
to  the  SAC  overall  xR  chart,  it  becomes  apparent  the 
importance  of  measuring  the  programs  performance  at  the 
lowest  level  possible  to  determine  specific  problems 
effecting  the  CIP.  Evaluation  of  individual  MDS  xR  charts 
identified  the  poor  cycle  time  performance  of  the  range  UCL . 
Evaluation  of  the  SAC  overall  xR  chart  indicated  a 
consistent  and  controlled  process.  Therefore,  evaluation  of 
the  SAC  overall  xR  chart  by  itself  would  not  have  identified 
that  individual  processes  were  out  of  control.  This 
observation  clearly  identifies  the  need  to  measure  the  CIP's 
performance  at  different  levels  in  order  to  determine 
specific  problem  areas  and  the  true  performance  of  the 
program  at  a  given  organizational  level. 

Correlation  of  Cycle  Time  to  Aircraft  Availability. 
Spearman's  Rank  Correlation  of  Coefficient  was  used  to 
determine  whether  a  correlation  existed  between  the  measure 
of  cycle  time  and  aircraft  availability  (total  MICAP  period 
hours).  The  WSMIS/Readiness  Assessment  Module  (RAM)  was 
accessed  to  extract  MICAP  period  hours  for  each  month  (May, 
June,  and  July  1991)  for  each  of  the  five  SAC  MDSs .  These 
values  were  then  summed  to  determine  an  overall  SAC  aircraft 
availability  value  (total  MICAP  period  hours,  see  Table  9). 


102 


Using  the  software  package  Statistix  3.1  (51),  Spearman’s 
Rank  Correlation  of  Coefficient  was  calculated  comparing  the 
aircraft  availability  values  to  the  average  cycle  time  for 
each  MDS  with  measurable  data  and  the  overall  SAC  cycle  time 
value.  Results  of  these  calculations  all  provided  a  zero 
correlation  between  MICAP  period  hours  and  cycle  time 
measurements . 


Table  9.  Aircraft  Availability  Data  to  be  Correlated 
to  Cycle  Time 


MICAP 

Per iod 

Hours 

May/ 91 

Jun/91 

Jul/91 

B-1 

278781 

300655 

224229 

C-135 

2977 

1898 

995 

B-52 

524028 

374151 

237212 

SAC  Overall 

805786 

676704 

462436 

NOTE;  MICAP  period  hours  are  total  MICAP  hours 
accumulated  for  each  item  within  a  weapon  system 
for  the  time  period  specified. 


To  further  investigate  the  possibility  of  correlation 
between  the  two  measures,  the  sum  of  cycle  time  hours  within 
each  MDS  was  also  compared  in  aggregate  to  MICAP  period 
hours.  Results  of  this  effort  generated  a  positive 
correlation  of  .5255.  In  order  to  reject  with  95% 
confidence  that  their  was  no  correlation,  a  value  greater 
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than  .6  was  required  to  support  the  existence  of  a 
correlation  (38:1212).  As  this  was  not  achieved,  this  study 
must  conclude  that  at  the  .05  level  of  significance  no 
correlation  exists  between  cycle  time  and  aircraft 
availability  (MICAP  period  hours). 

The  lacJ<  of  correlation  between  cycle  time  and  aircraft 
availability  suggests  that  the  CIP  may  not  be  managing  the 
correct  items.  Had  a  positively  correlated  value  resulted 
within  the  specified  level  of  significance,  then  evidence  of 
effective  management  of  the  CIP  might  have  been  concluded. 

Once  again,  the  number  of  data  points  (3)  used  in  the 
calculation  of  individual  MDS  correlation  values  does  not 
provide  substantial  confidence  in  their  outcomes. 

Generally,  the  greater  the  number  of  data  pairs  available  to 
correlate,  the  stronger  the  outcome  of  the  test.  Thus, 
findings  for  individual  MDS  correlations  must  be  caveated 
with  the  need  for  additional  calculation  of  correlation  as 
more  data  become  available  in  the  future. 

Survey  of  Critical  Item  Managers.  In  order  to 
determine  the  potential  of  percent  defects  and  cycle  time 
for  evaluation  of  the  CIP,  critical  item  managers  from 
several  levels  within  the  Air  Force  were  presented  a  written 
pac)(age  (see  Appendix  B)  explaining  the  measures,  and  then 
subsequently  asked  specific  questions  addressing  the 
practicality  for  field  use  (see  Appendix  C) .  Of  tie  23 
managers  initially  contacted,  22  managers  completed  the 
interview  process.  Based  on  these  completed  interviews. 
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critical  item  manager's  opinions  o£  the  practical  use  of 
percent  defects  and  cycle  time  were  determined.  Table  10 
synopsizes  the  results  of  these  interviews- 

As  indicated  in  Table  10,  a  large  majority  of  the 
managers  interviewed  (73%  for  percent  defects  and  68%  for 
cycle  time)  recommended  these  measures  be  considered  for 
field  application.  The  remaining  managers  interviewed 
either  did  not  support  the  use  of  these  measures  or  had  no 
opinion  at  the  time.  The  reasons  for  recommending  or  not 
recommending  these  measures  were  gathered  during  the 
interviews  using  questions  contained  in  Appendix  C.  To 
further  investigate  management  opinion  of  these  measures, 
reasons  for  support  and  non  support  of  these  measures  are 
discussed  below. 

Reasons  for  SuPDortina  the  Use  of  Percent  Defects. 
A  consistent  reason  for  supporting  the  use  of  percent 
defects  to  evaluate  the  CIP's  performance  was  the  current 
lack  of  any  measurement  or  evaluation  capability  of  the 
program.  Thirteen  managers  expressed  frustration  with  the 
program  because  there  was  currently  no  means  to  receive 
feedback  as  to  whether  management  actions  that  were 
implemented  were  effective.  Therefore,  managers  generally 
welcomed  any  tool  that  would  provide  feedback  on  the 
program's  performance. 

Another  popular  reason  supporting  the  use  of  percent 
defects  was  that  it  identified  and  highlighted  the  CIP's 
problem  areas.  Thirteen  managers  stated  that  by  using  this 
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Table  10.  Management  View  of  Percent  Defects/Cycle  Time 


%  Defects  Cycle  Time  It 


Interviewed 

Position 

Rec 

No 

Rec 

No 

Opn 

Rec 

No 

Rec 

No 

Opn 

Division 

Chiefs 

4 

0 

1 

2 

1 

2 

5 

SPM 

3 

2 

0 

4 

1 

0 

5 

CIM 

3 

1 

0 

3 

1 

0 

4 

HQ  AFLC 

1 

1 

0 

1 

1 

0 

2 

HQ  USAF 

0 

0 

1 

0 

0 

1 

1 

MAJCOM 

4 

0 

0 

4 

0 

0 

4 

AFAA 

1 

0 

0 

1 

0 

0 

1 

Total 

16 

4 

2 

15 

4 

3 

22 

% 

73 

18 

9 

68 

18 

14 

100 

measure,  they  could  determine  if  the  correct  items  were 
being  identified.  By  looking  at  the  three  defect 
conditions,  managers  maintained  they  could  determine  where 
the  program  was  identifying  the  right  items  and  where  it  was 
not . 

Six  managers  also  felt  that  percent  defects  could 
communicate  how  well  the  automated  and  manual  critical  item 
lists  matched  up.  Managers  maintain  that  WSMIS,  a  very 
expensive  Air  Force  decision  support  system,  is  not  being 
fully  used  due  to  the  lack  of  confidence  in  its  products. 

In  order  to  improve  confidence  in  WSMIS  products,  problem 
areas  must  be  identified  and  corrected.  Defect  condition 
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one,  which  identifies  those  items  in  the  CIP  that  were  not 
recommended  as  candidates  by  WSMIS/GWAM,  provides  such  a 
comparison  capability. 

Eleven  managers  identified  the  ability  of  percent 
defects  to  communicate  whether  the  criteria  were  being 
followed  or  not.  In  general,  items  submitted  to  be  entered 
in  the  CIP  should  meet  the  preestablished  criteria. 

However,  managers  maintain  that  there  are  special 
circumstances  which  justify  entry  of  items  into  the  CIP  that 
do  not  meet  MAJCOM  criteria.  To  ensure  that  the  system  is 
not  being  abused,  managers  felt  that  identification  of  items 
entered  into  the  CIP  that  do  not  meet  criteria  would  allow 
further  investigation  into  the  legitimacy  of  these  actions. 
Defect  condition  two,  identification  of  items  not  fulfilling 
MAJCOM  criteria,  was  identified  by  managers  as  providing 
this  analysis  capability. 

One  manager  believed  identification  of  seasonal  trends 
was  possible  through  tracking  of  percent  defects.  As 
identification  of  seasonal  trends  could  be  very  beneficial 
to  supportabi 1 i ty  improvements,  this  manager  supported  the 
use  of  percent  defects. 

Seven  managers  identified  that  use  of  percent  defects 
would  aid  in  determining  whether  the  critical  item  selection 
criteria  were  correct.  Managers  claimed  that  this  type  of 
capability  was  needed  in  order  to  enable  corrective  actions 
to  the  existing  program  and  build  confidence  in  the  products 
the  program  generates. 
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Five  managers  stated  that  the  use  of  percent  defects 
would  enhance  the  communication  and  coordination  between 
support  and  operational  commands  on  CIP  objectives. 

Manager's  felt  that  common  objectives  and  a  management 
checks  and  balance  capability  were  necessary  to  get  the 
program  back  on  track.  They  perceived  percent  defects  would 
be  able  to  provide  such  a  capability. 

In  short,  managers  supporting  the  use  of  percent 
defects  communicated  the  need  for  a  quantifiable  feedback 
mechanism  which  identified  specific  problem  areas  and 
established  common  objectives  for  all  managers  involved  in 
the  program.  Based  on  these  needs  and  their  perception  of 
how  percent  defects  could  be  applied  to  the  CIP,  it  was 
their  opinion  that  percent  defects  could  be  instrumental  in 
improving  management  of  the  program. 

Reasons  for  Not  Supporting  the  Use  of  Percent 
Defects .  Of  the  18%  not  supporting  the  use  of  percent 
defects,  one  reason  given  against  its  use,  was  the 
additional  workload  that  it  would  generate.  One  non¬ 
supporting  manager  stated  that  implementation  of  the  Total 
Quality  Management  (TQM)  philosophy  (perceived  by  this 
manager  as  producing  a  better  product  with  less  people),  was 
not  consistent  with  the  creation  of  additional  reporting  and 
administrative  requirements.  Two  other  non-supporting 
managers  emphasized  the  lack  of  resources,  including  money 
and  personnel,  necessary  to  support  use  of  these  measures. 
These  managers  emphasized  that  the  current  lack  of  resources 


108 


was  already  affecting  proper  management  of  the  program  and 
that  any  additional  administrative  requirements  would  just 
add  to  the  existing  problem. 

Another  reason  managers  did  not  support  using  percent 
defects  was  the  current  performance  of  the  automated  and 
manual  management  systems.  Three  managers  with  this  concern 
pointed  out  several  shortfalls  in  this  area.  First,  these 
managers  claimed  that  the  automated  system  is  not  real-time 
enough  to  allow  a  high  degree  of  confidence  in  WSMIS/GWAM 
critical  item  candidates.  These  managers  maintain  that 
other  automated  systems  that  feed  WSMIS/GWAM  provide 
outdated  data,  resulting  in  the  wrong  items  being  identified 
as  candidates.  Concerning  the  manual  management  system, 
managers  identified  that  pre-established  criteria  were  not 
always  followed  by  the  MAJCOMs  in  selection  of  critical 
items.  This  disregard  for  the  criteria,  though  many  times 
justified,  is  not  subject  to  evenly  weighted  negotiation 
between  MAJCOM  and  ALC  managers.  Therefore,  if  the  MAJCOMs 
demand  that  items  not  conforming  to  criteria  be  added  to  the 
CIP,  ALC  managers  must  comply.  In  their  opinion,  this  would 
result  in  an  inflated  percent  defects  measurement,  making 
the  measurement  less  accurate  and  less  useful. 

Two  managers  felt  that  percent  defects,  though 
providing  an  overall  evaluation  of  the  program,  did  not 
provide  enough  detail  to  correct  the  problems  occurring  in 
the  individual  items  themselves.  These  managers  claimed 
that  in  order  to  truly  be  useful,  measures  that  are  used 
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must  be  able  to  point  to  those  items  causing  the  problems  in 
addition  to  what  the  problems  are.  Some  of  these  managers 
confided  that  if  percent  defects  could  identify  the  problems 
occurring  to  the  individual  items  verses  the  overall 
program,  it  would  then  be  a  useful  measure. 

A  common  reason  for  not  recommending  che  use  of  percent 
defects  was  the  additional  reporting  requirement  that  it 
would  generate.  Managers  with  this  opinion  (mainly  ALC 
positions)  were  concerned  with  having  to  track  and  report  to 
higher  management  on  items  for  which  they  had  little 
control.  Because  selection  of  critical  items  rests 
ultimately  with  the  MAJCOMs,  these  managers  felt  their 
tracking  and  reporting  to  higher  management  would  be 
fruitless  given  the  ALC's  minimal  say  in  selection  of 
critical  ’ terns . 

In  summary,  managers  not  supporting  the  use  of  percent 
defects  in  evaluation  of  the  CIP  based  their  opinions  on  the 
current  lack  of  real-time  operation  of  the  automated  system, 
MAJCOM  nonconcurrence  to  pre-established  critical  item 
selection  criteria,  the  existing  lack  of  money  and  personnel 
resources  necessary  to  accomplish  administrative 
requirements,  and  the  additional  workload  that  would  be 
placed  on  critical  item  managers  resulting  in  little  or  no 
gain  in  effective  management  of  the  program. 

Reasons  for  Supporting  the  Use  of  Cycle  Time.  Of 
the  22  managers  interviewed,  68%  supported  the  use  of  cycle 
time  in  evaluation  of  the  CIP’s  performance.  With  this  in 
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mind,  it  should  be  noted  that  all  managers  both  supportive 
and  non-supportive  of  cycle  time  as  a  useful  measure, 
identified  that  certain  items  managed  by  the  program  would 
be  outliers  in  cycle  time  measurement.  The  reasons  for  this 
include  pending  weapon  system  modifications,  contract  lead 
times  and  other  extenuating  circumstances  beyond  critical 
item  manager  control  which  inherently  drive  up  item  cycle 
time.  In  spite  of  this,  several  reasons  were  identified  for 
why  cycle  time  would  be  useful  to  the  management  of  the  CIP. 

One  such  reason  was  to  provide  management  with  an 
average  measure  for  aid  in  establishing  how  long  an  item 
should  remain  in  the  program  before  it  is  expected  to  become 
well.  Though  some  items  will  stay  in  the  program  for  long 
periods  for  reasons  stated  above,  fifteen  managers  believed 
an  average  cycle  time  measure  would  provide  them  a  basis  for 
measuring  individual  item  performance.  This  would  enable 
them  to  identify  those  items  that  have  been  in  the  program  a 
long  time  which  do  not  have  a  valid  justification  unlike 
those  reasons  stated  above. 

Others  felt  that  cycle  time  would  provide  a  good 
overall  evaluation  of  how  the  program  was  performing  as  a 
whole.  These  managers  believed  that  cycle  time  would  be 
especially  useful  in  providing  a  top  management  perspective 
of  the  program's  performance. 

Two  lower  level  managers  identified  the  usefulness  of 
cycle  time  in  improving  get-well  plan  development  and  depot 
funding  process  times.  By  using  cycle  time,  these  managers 
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believed  more  improvement  could  be  accomplished  by 
decreasing  times  involved  in  administrative  actions  and 
thereby  speed  the  start  of  other  process  actions  necessary 
in  item  get-well  attainment. 

Another  management  benefit  identified  with  using  cycle 
time  was  the  ability  to  show  customers  (MAJCOMs)  how  support 
has  been  improved.  According  to  six  managers,  use  of  cycle 
time  measurement  could  quantitatively  communicate  the 
benefits  of  using  the  CIP  by  comparing  CIP  item  cycle  time 
to  the  expected  lead  times  of  items  acquired  through 
standard  channels. 

Two  managers  believed  that  the  use  of  cycle  time  could 
enhance  the  ability  to  obtain  required  funding  necessary  to 
enable  the  CIP  to  operate  properly.  These  managers  stated 
that  the  CIP,  though  widely  believed  to  be  100%  funded,  was 
only  funded  at  about  60%  -  70%.  These  managers  also 
maintained  that  this  shortfall  prevented  the  effective 
management  of  the  program  resulting  in  its  current 
performance.  These  managers  believed  that  the  use  of  cycle 
time  as  a  measure  would  help  communicate  the  need  for 
additional  funding  by  quantitatively  and  objectively 
evaluating  the  current  program's  performance. 

In  review,  managers  who  support  the  use  of  cycle  time 
in  evaluating  the  CIP  believe  it  will  provide  several 
benefits.  These  benefits  include  the  establishment  of  CIP 
average  processing  times  for  use  as  bench  marks,  an 
evaluation  capability  for  the  overall  program,  a  means  by 
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which  to  improve  administrative  and  other  processing  times, 
a  measure  useful  in  communicating  customer  support  levels, 
and  a  tool  for  use  in  obtaining  required  funding  levels. 

Reasons  for  Not  Supporting  the  Use  of  Cycle  Time. 
Of  the  18%  of  managers  who  did  not  like  cycle  time  as  a 
measure,  basically  four  reasons  were  given.  The  first  and 
most  common  was  the  concern  about  outliers.  As  previously 
identified,  some  items  exist  which  take  an  extensive  amount 
of  time  (years  in  some  instances)  to  get  well.  Managers  not 
supporting  the  use  of  cycle  time  maintain  that  these 
outliers  will  drive  up  the  overall  cycle  time  average  and 
make  the  measure  useless. 

Along  the  same  thought,  one  manager  identified  that  the 
aggregate  measure  established  through  the  use  of  cycle  time 
does  not  provide  enough  information  to  work  individual 
items.  The  manager  maintained  that  this  shortfall  prevents 
cycle  time  from  being  considered  useful  as  it  communicates 
that  a  problem  exists  but  not  exactly  where. 

Another  critical  item  manager  felt  that  cycle  time  was 
not  useful  because  it  was  evaluated  too  late  in  the  process. 
This  manager  believed  that  items  must  be  identified  and 
managed  before  they  become  critical  otherwise  the  time 
required  to  make  them  well  becomes  unacceptably  long.  It 
was  the  belief  that  a  measure  must  be  implemented  which 
provided  a  more  proactive  approach. 

Finally,  one  manager  stated  that  cycle  time  would  not 
be  a  useful  measure  due  to  insufficient  funding.  This 


113 


manager  maintained  that  items  that  remained  in  the  program 
for  long  periods  of  time  were  there  due  to  a  lack  of  funds 
to  properly  correct  the  problem  causing  such  items  to  be 
critical.  They  maintained  that  because  insufficient  funding 
was  available,  identification  of  poor  performance  was 
useless  as  no  action  could  be  taken  to  correct  the 
situation. 

In  short,  managers  not  supporting  the  use  of  cycle  time 
as  a  measure  of  the  CIP's  performance  based  their  opinions 
on  four  things.  These  reasons  were  that  too  many  long  lead 
time  items  were  present  in  the  program,  an  aggregate  measure 
was  not  useful  in  individual  item  management,  cycle  time  was 
not  consistent  with  a  proactive  approach,  and  insufficient 
funding  for  the  program  made  measurement  of  cycle  time 
useless . 

Other  Survey  Findings.  In  completing  the 
interview  process,  significant  caveats  and  concerns  were 
discussed  with  the  critical  item  managers  interviewed.  As 
these  conditions,  in  some  cases,  determined  whether  or  not 
the  manager  did  or  did  not  recommend  the  use  of  either  of 
the  two  measures,  disclosure  of  the  caveats  is  important. 

In  addition,  discussion  of  relevant  concerns  surrounding 
percent  defects  and  cycle  time  and  how  they  might  be 
implemented  are  also  important. 

A  caveat  expressed  by  the  vast  majority  of  managers, 
was  the  need  to  have  the  measurement  process  automated. 
Managers  emphasized  that  manual  calculation  of  these 
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measures  would  be  infeasible  given  current  manning  levels 
and  work  loads.  Several  managers  were  averse  to  even  10  - 
15  minute  increases  in  workload  necessary  to  post  automated 
measurement  calculations  to  administrative  forms.  One 
manager  who  did  not  recommend  the  use  of  these  two  measures, 
did  support  them  in  theory;  however,  he  did  not  recommend 
their  use  due  to  the  need  to  spend  money  in  other  areas  of 
the  program  verses  developing  an  automated  measurement 
capability.  On  the  other  hand,  numerous  managers  who  did 
support  use  of  the  two  measures,  based  their  recommendation 
on  the  condition  that  the  process  would  be  automated. 

Several  concerns  were  also  voiced  by  managers 
addressing  how  the  measures  might  be  used.  Among  these 
concerns,  managers  wanted  to  know  at  what  level (s)  of  the 
Air  Force  that  these  measures  would  be  used.  In  addition, 
several  managers  wanted  to  know  if  there  would  be  a  specific 
organization  who  would  be  responsible  for  monitoring 
performance  of  the  CIP  if  the  measures  were  implemented. 
Lower  level  managers  were  concerned  that  the  measures  would 
be  used  against  them  to  make  corrective  actions  that  were 
out  of  their  control  due  to  misunderstanding  by  upper 
management.  Because  of  this  concern  alone,  several  lower 
level  managers  did  not  recommend  the  use  of  either  of  the 
two  measures. 

Throughout  all  management  levels  of  critical  item 
managers  contacted  during  this  study,  there  was  a  consistent 
conclusion  that  the  CIP  was  not  functioning  as  it  was 
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designed.  As  no  formal  measures  exist  to  evaluate  the 
program,  only  subjective  opinions  could  be  gathered 
concerning  its  performance.  The  majority  of  these  opinions 
clearly  indicated  that  the  overall  CIP's  performance  was 
considered  poor.  The  emphasis  placed  on  particular  CIP 
problems  varied  depending  on  the  type  of  organization 
contacted . 

In  general,  managers  in  support  organizations 
emphasized  the  lack  of  funding  as  driving  the  poor 
performance  of  the  CIP.  They  maintained  that  their  ability 
to  properly  correct  problem  items  added  to  the  CIP  was 
directly  related  to  funding  levels  received.  As  they 
explained,  the  reason  that  expensive  long  lead  time  items 
remain  in  the  CIP  for  years,  is  that  only  a  few  assets  may 
be  corrected  in  a  given  period  due  to  the  inadequate 
funding.  These  managers  believed  that  given  proper  funding, 
recurring  and  expensive  items  identified  as  critical  could 
be  managed  more  effectively. 

Managers  from  lower  level  support  organizations 
emphasized  the  need  to  decrease  work  load,  streamline 
administrative  functions,  and  have  a  stronger  voice  in 
management  of  the  program.  These  lower  level  support 
managers  also  felt  burdened  for  having  responsibility  to 
correct  a  problem  for  which  they  had  little  or  no  control. 
Two  lower  level  support  managers  perceived  the  use  of 
percent  defects  and  cycle  time  as  the  creation  of  another 
reporting  requirement  for  which  they  would  have  to  provide 
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answers  to  questions  they  currently  are  not  responsible  for. 
Other  support  organization  managers  expressed  frustration 
because  they  have  no  means  to  determine  if  their  current 
efforts  within  the  program  were  of  any  benefit  to  their 
customers.  In  short,  there  was  a  divergence  of  opinion 
among  lower  level  support  managers.  The  majority  of  these 
managers  agreed  that  a  measurement  capability  was  desirable. 
On  the  other  hand,  40%  of  the  managers  at  this  level  were 
against  increased  administrative  workload  that  would  result 
from  implementation  of  the  measures 

Managers  from  operational  organizations  emphasized  the 
need  to  identify  the  correct  items,  expedite  corrective 
actions,  and  improve  inter-organizational  communication. 
MAJCOM  responses  indicated  a  more  urgent  need  for  feedback 
on  the  CIP's  performance  in  order  to  improve  current  process 
limitations.  Of  all  the  managers  interviewed,  it  was 
operational  managers  who  expressed  the  strongest  desire  to 
see  an  evaluation  capability  for  the  CIP  developed  and 
implemented . 

Chapter  IV  Summary 

In  this  chapter,  findings  resulting  from  implementation 
of  Chapter  Ill's  methodology  were  presented.  Included  in 
these  findings  were  the  developed  control  charts  for  percent 
defects  and  cycle  time,  results  of  Spearman's  Rank 
Correlation  of  Coefficient  calculations  and  the  opinions  of 
experienced  critical  item  managers  addressing  the 
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practicality  of  percent  defects  and  cycle  time  for  field 
use.  Chapter  V  presents  the  conclusions  and  recommendations 
resulting  from  the  information  and  analysis  presented  in 
Chapters  I  -  IV. 
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V. 


Conclusions  and  Recommendations 


Introduction 

The  purpose  of  this  research  was  to  examine  the 
potential  of  the  two  measures  percent  defects  and  cycle  time 
to  evaluate  the  Air  Force  Critical  Item  Program  (CIP).  To 
accomplish  this  task,  actual  data  were  collected  and  control 
charts  constructed  to  demonstrate  the  potential  of  the  two 
measures  identified  above.  The  research  also  compared 
percent  defects  and  cycle  time  measures  to  aircraft 
availability  measures  to  determine  whether  a  correlation 
existed  between  them.  Finally,  an  introductory  package 
(Appendix  B)  describing  percent  defects  and  cycle  time  was 
presented  to  22  experienced  critical  item  managers  from 
various  levels  and  positions  within  the  Air  Force.  After 
reviewing  this  package,  critical  item  managers  were  asked 
several  questions  (Appendix  C)  addressing  the  potential  of 
these  measures  for  field  use. 

Conclus ions 

As  outlined  in  Chapter  I  and  reiterated  in  Chapter  III, 
two  investigative  questions  were  examined  to  address  the 
requirements  posed  in  this  study's  problem  statement.  To 
thoroughly  research  each  investigative  question,  several 
associated  measurement  questions  were  also  addressed. 

Having  completed  the  findings  portion  of  this  study, 
sufficient  information  now  exists  to  make  conclusions 
regarding  investigative  and  measurement  questions. 
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Investigative  Question  1.  Investigative  question  1 
asks,  "Does  measurement  of  percent  defects  aid  the  Air  Force 
in  managing  the  CIP  effectively?"  In  addressing  this 
question,  five  measurement  questions  were  also  asked.  From 
these  measurement  questions  the  following  conclusions  were 
reached , 

In  answering  measurement  question  1,  it  was  not 
possible  to  compare  items  added  to  the  CIP  that  were  not 
recommended  by  WSMIS/GWAM.  As  a  result,  data  were  not 
available  to  quantitatively  answer  whether  items  in  the  CIP 
met  MAJCOM  criteria.  However,  the  description  of  this 
defect  condition  was  included  in  Appendix  B  and  was 
presented  to  experienced  critical  item  managers.  Of  these 
managers,  the  majority  supported  the  need  to  consider  this 
defect  condition  in  measurement  of  the  CIP’s  performance. 
This  concern  by  managers  supports  the  prospect  that  items 
are  currently  in  the  program  that  do  not  meet  MAJCOM 
criteria.  While  support  from  these  managers  is  encouraging, 
the  data  should  still  be  evaluated  to  more  completely  assess 
this  important  issue. 

In  addressing  measurement  question  2,  quantitative  data 
were  available.  As  demonstrated  by  the  p  control  charts 
exhibited  in  Chapter  IV,  items  in  the  CIP  were  not  found  to 
consistently  correspond  to  items  identified  by  WSMIS/GWAM. 

In  comparing  these  two  lists,  an  average  defect  rate  of  50% 
was  found.  This  value  confirms  that  items  in  the  CIP  do  not 
acceptably  correspond  to  items  recommended  by  WSMIS/GWAM. 
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Analysis  of  measurement  question  3  was  limited  to  only 
one  quarter's  data.  For  this  quarter  (ending  the  month  of 
April  1991),  all  items  contained  on  the  list  were  reviewed 
to  determine  if  they  had  been  entered  and  deleted  from  the 
CIP  before.  As  a  result  of  this  analysis,  an  average  of  34% 
of  the  items  in  the  program  in  April  1991  were  recurring 
items.  This  high  percentage  suggests  that  many  of  the  same 
items  do  enter  and  leave  the  CIP  on  a  recurring  basis. 

The  Spearman's  Rank  Correlation  of  Coefficient  values 
were  calculated  for  percent  defects  measures,  including  all 
defect  conditions  for  each  MDS,  and  aircraft  availability. 
The  results  of  these  calculations  ranged  from  positive  ,80, 
to  negative  .80.  The  overall  correlation  value  calculated 
was  negative  .40.  A  value  of  .9  or  greater  was  required  to 
substantiate  a  correlation  within  a  .05  level  of 
significance.  As  this  was  not  achieved,  no  correlation  was 
concluded  for  these  calculations.  Ah  aggregate  correlation 
of  the  combined  MDSs  was  also  accomplished.  This  resulted 
in  a  positive  correlation  of  .3033;  however,  a  value  greater 
than  .377  was  required  to  substantiate  a  correlation  at  the 
.05  level  of  significance.  Again,  the  conclusion  that  no 
correlation  existed  was  reached  for  a  .05  level  of 
significance.  To  attempt  to  compensate  for  the  inability  to 
measure  all  defect  conditions  in  each  quarter,  percent 
defects  measures  for  defect  condition  one  (which  was 
measured  in  all  four  quarters)  was  also  aggregately 
correlated  against  aircraft  availability  (MICAP  period 
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hours)  for  each  MDS .  The  result  of  this  calculation 
generated  a  correlation  value  of  ,3341.  As  with  the 
original  test,  a  value  greater  than  .377  was  required  to 
substantiate  the  existence  of  a  correlation  with  a  .05  level 
of  significance.  Once  again,  it  was  determined  that  no 
correlation  existed  with  a  .05  level  of  significance. 

Further  analysis  showed  that  no  correlation  existed  between 
MICAP  period  hours  and  the  number  of  items  entered  into  the 
CIP  for  a  given  weapon  system.  Correlation  calculations  for 
this  test  resulted  in  a  correlation  value  of  .2031;  however, 
a  value  greater  than  .49  was  required  to  establ 
significance  at  the  .05  level.  These  findings  all 
support  that  the  wrong  items  are  being  managed  in  the  CIP. 

Due  to  the  use  of  partial  percent  defects  data  for  two 
of  the  four  periods  analyzed,  the  confidence  in  these 
results  is  weakened  to  a  small  degree.  Nevertheless,  this 
study  concludes  that  no  correlation  exists  between  percent 
defects  and  MICAP  period  hours  or  for  MICAP  period  hours  and 
the  number  of  items  entered  into  the  CIP. 

A  survey  of  22  critical  item  managers  was  completed 
using  information  and  questions  contained  in  Appendices  B 
and  C.  Results  of  this  process  were  summarized  in  Table  9. 
The  high  percentage  of  managers  recommending  the  use  of 
percent  defects  (73%),  strongly  suggests  that  critical  item 
managers  find  this  measure  useful.  Of  all  managers 
interviewed,  only  18%  of  the  managers  did  not  recommend  the 
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use  of  percent  defects.  The  remaining  managers  (9%)  had  no 
opinion . 

From  the  findings  stated  above  concerning  individual 
measurement  questions,  several  conclusions  can  be  reached. 
First,  managers  believe  that  items  are  entering  the  CIP  that 
do  not  meet  MAJCOM  criteria.  Second,  items  recommended  by 
WSMIS/GWA.M  do  not  consistently  correspond  to  items  actually 
entered  into  the  CIP.  Third,  over  30%  of  those  items 
evaluated  were  identified  as  having  entered  and  left  the  CIP 
at  a  previous  date.  Fourth,  current  calculations  indicate 
that  there  is  no  correlation  between  aircraft  availability 
and  percent  defects  or  the  number  of  items  entered  into  the 
CIP  at  the  .05  level  of  significance.  Fifth,  73%  of 
critical  item  managers  interviewed  supported  the  further 
development  of  percent  defects  for  field  use.  Based  on 
these  determinations,  this  study  concludes  that  the 
measurement  of  percent  defects  could  aid  the  Air  Force  in 
managing  the  CIP. 

Investigative  Question  2.  Investigative  question  2 
asks,  "Does  measurement  of  cycle  time  aid  the  Air  Force  in 
managing  the  CIP  effectively?"  In  addressing  this  question, 
three  measurement  questions  were  also  asked.  From  these 
measurement  questions  the  following  conclusions  were 
reached . 

Cycle  time  was  measured  by  looking  at  WSMIS/GWAM 
Add/Delete  reports.  The  value  for  "months  critical"  was 
averaged  within  each  of  the  five  MDSs  for  items  shown  as 
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deleted  on  these  reports.  Fron  these  data,  both  average  and 
range  information  were  gathered.  This  information  was  then 
consolidated  at  the  MAJCOM  level  producing  a  SAC  overall 
cycle  time  measurement.  As  a  result  of  this  effort, 
individual  MDS  cycle  time  measures  were  found  to  cover  from 
3.9  to  10  months,  while  rar.je  measures  within  an  MDS  covered 
from  7  to  18  months.  Overall  MAJCOM  values  for  cycle  time 
ranged  from  6.1  to  6.4  months  for  average  and  6.3  to  6.4 
months  for  range.  Within  each  MDS,  little  variation  was 
noticed  for  average  values  and  all  were  within  UCL  control 
limits.  However,  range  values  did  show  variation  and  all 
meas  red  MDSs  exceeded  the  established  UCL.  Between  MDSs 
there  was  considerably  more  variation  (from  3.9  to  10  montns 
for  average  values  and  7  to  13  months  for  range  values). 
Addressing  measurement  question  6,  within  each  MDS  there  was 
very  poor  cycle  time  performance.  For  the  MAJCOM  as  a 
whole,  cycle  time  performance  appeared  to  be  in  control. 
These  measurements  established  the  importance  of  multi-level 
evaluation  and  provided  a  basis  for  further  use  of  these 
measures  with  SAC  weapon  systems  in  the  future. 

As  was  the  case  with  percent  defects,  very  few  data 
periods  were  available  to  calculate  Spearman's  Rank 
Correlation  of  Coefficient  between  cycle  time  and  aircraft 
availability  (three  periods.  May,  June,  and  July  1991). 
Resulting  calculations  provided  a  zero  correlation  between 
these  two  measures  for  both  individual  MDSs  and  SAC  overall. 
Aggregate  correlation  calculations  using  the  sum  of  cycle 
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time  data  and  the  total  MICAP  period  hours  generated  a 
positive  correlation  of  .5255.  However,  a  value  of  greater 
than  .6  wa»  required  to  substantiate  the  existence  of  a 
correlation  within  a  .05  level  of  significance.  As  this  was 
not  attained,  the  resulting  conclusion  was  again  that  no 
correlation  existed  at  a  .05  level  of  significance.  Based 
on  these  results,  management  must  conclude  at  this  time  that 
there  is  no  correlation  between  aircraft  availability  and 
cycle  time  at  the  .05  level  of  significance. 

As  stated  above,  survey  of  22  critical  item  managers 
was  completed  using  information  and  questions  contained  in 
Appendices  B  and  C.  Results  of  this  process  were  summarized 
in  Table  9.  The  percentage  of  managers  recommending  the  use 
of  cycle  time  (68%),  suggests  that  critical  item  managers 
find  this  measure  useful.  Of  all  managers  interviewed,  only 
18%  of  the  managers  did  not  recommend  cycle  time  be  used. 

The  remaining  managers  (14%)  had  no  opinion. 

From  the  findings  stated  above  addressing  individual 
measurement  questions,  several  conclusions  have  been 
reached.  First,  cycle  time  measurements  were  out  of  control 
within  each  MDS,  however,  they  were  in  control  for  SAC 
overall.  Second,  correlation  values  for  cycle  time  and 
aircraft  availability  indicated  no  correlation  at  a  .05 
level  of  significance.  Third,  68%  of  the  critical  item 
managers  interviewed  supported  the  further  development  of 
cycle  time  as  a  measure  of  the  CIP’s  performance.  Based  on 
these  determinations,  this  study  concludes  that  the 


125 


measurement  of  cycle  time  could  aid  the  Air  Force  in 
managing  the  CIP  more  effectively. 

Overall  Objective.  The  objective  of  this  study  was  to 
test  the  proposition  that  the  measures  of  percent  defects 
and  cycle  time  could  provide  the  Air  Force  with  a  useful 
tool  for  evaluating  the  performance  of  the  CIP.  In  reaching 
this  objective,  actual  data  were  gathered  and  measured  using 
percent  defects  and  cycle  time  with  their  respective  control 
charts.  The  correlation  of  these  measures  to  aircraft 
availability  was  also  determined.  Finally,  res  ils  of  the 
above  efforts  were  presented  to  experienced  critical  item 
managers  in  the  field  to  determine  the  potential  of  the  two 
measures  for  field  use.  Implementation  of  the  above 
research  resulted  in  the  conclusion  that  both  percent 
defects  and  cycle  time  measures  could  be  useful  in  the 
effective  management  of  the  CIP.  Though  the  use  of  partial 
data  weakened  the  confidence  in  correlatioh  results,  applied 
measurement  of  the  data  and  subsequent  interview  of  critical 
item  managers  substantiated  the  usefulness  of  these 
measures . 

A  clear  outcome  of  this  research  was  the  need  for  some 
form  of  evaluation  for  the  CIP's  performance.  Managers  at 
every  level  fully  supported  the  development  of  some  kind  of 
measurement  technique  and  perceived  it  as  necessary  in  order 
to  improve  the  current  performance  of  the  program.  All 
managers  agreed  that  they  needed  some  form  of  feedback  to 
determine  when  and  if  corrective  actions  taken  are 
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effective.  Therefore,  it  is  the  overall  conclusion  of  this 
study  that  both  percent  defects  and  cycle  time  could  provide 
the  Air  Force  with  a  useful  tool  for  evaluating  the 
performance  of  the  CIP. 

Recommendations 

A  test  case  should  be  performed  in  an  operational 
environment  to  validate  the  functionality  of  these  two 
measures  in  relation  to  the  CIP.  This  test  would  provide  a 
broader  understanding  of  the  mechanics  needed  to 
successfully  implement  these  measures  Air  Force  wide.  In 
addition,  an  operational  test  would  provide  an  actual 
determination  of  the  usefulness  of  these  two  measures  in 
real  management  of  the  CIP. 

This  study  also  recommends  that  cycle  time  objectives 
be  established  for  items  managed  within  the  CIP.  Current 
lack  of  cycle  time  objectives  has  resulted  in  many  items 
remaining  in  the  program  for  years  at  a  time  without 
receiving  the  proper  management  attention.  Establishment  of 
cycle  time  objectives  would  provide  a  baseline  time  for 
getting  items  through  the  program  and  a  tool  for  managers  of 
extreme  problem  items  to  receive  additional  resources. 

Further  development  of  automated  tracking  and 
measurement  capabilities  are  also  necessary.  Several 
shortfalls  in  the  current  automated  system  must  be  corrected 
in  order  to  institute  the  measures  of  percent  defects  and 
cycle  time.  The  enhancement  and  development  of  automated 
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capabilities  was  emphatically  stressed  by  all  managers,  due 
to  the  labor  and  time  consuming  effort  required  to  apply 
these  measures  through  manual  methods.  Without  automation, 
managers  stated  they  would  not  support  the  use  of  the  two 
measures  due  to  insufficient  resources.  Specific  details 
are  identified  in  the  following  paragraphs. 

The  first  automated  improvement  should  be  the  ability 
to  determine  the  cycle  time  of  items  currently  maintained  in 
the  CIP.  As  identified  in  Chapters  III  and  IV,  the  current 
system  is  only  able  to  aggregately  report  cycle  times  on 
items  deleted  from  the  program.  In  order  to  determine  true 
cycle  time  measurement  and  prevent  management  work-arounds 
which  divert  adverse  attention,  the  cycle  time  of  items 
currently  maintained  in  the  CIP  as  well  as  items  deleted 
from  the  CIP  must  be  able  to  be  determined. 

The  second  automated  improvement  required  is  the 
ability  to  compare  items  recommended  as  candidates  by 
WSMIS/GWAM  to  those  actually  entered  into  the  CIP.  This  is 
necessary  to  determine  how  effectively  GWAM  is  identifying 
those  items  perceived  as  most  critical  by  the  MAJCOMs.  This 
would  aid  in  improving  both  automated  and  manual  processes. 
The  value  of  this  capability  for  the  automated  process  would 
include  determining  the  appropriateness  of  the  criteria  and 
the  accuracy  of  the  data  within  WSMIS.  The  value  to  the 
manual  process  would  include  decreased  workload  in  the 
current  manual  correlation  of  these  two  lists.  Also,  as  the 
CIP  begins  to  improve  and  more  confidence  develops  in  GWAM 
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products,  MAJCOM  workloads  would  be  significantly  decreased 
due  to  fewer  resources  being  used  in  manual  tracking  of  the 
program. 

The  third  automated  improvement  required  is  the  ability 
to  determine  the  criteria  status  of  items  in  the  CIP  which 
are  not  recommended  as  candidates  by  the  WSMIS/Get  Well 
Assessment  Module  (GWAM).  This  capability  must  be 
established  to  determine  if  items  not  recommended  by  GWAM 
and  entered  into  the  CIP  are  truly  critical.  Even  under 
ideal  conditions,  some  items  will  have  justification  to  be 
in  the  program  even  though  they  were  not  recommended  by 
GWAM.  The  proposed  option  would  readily  enable  the  MAJCOMs 
and  SPMs  to  double  check  these  type  items  to  ensure  their 
addition  to  the  CIP  was  properly  justified. 

The  fourth  automated  improvement  required  is  the 
ability  to  easily  identify  items  that  are  recurring  within 
the  CIP.  A  time  frame  would  have  to  be  established  so  that 
WSMIS  could  search  backward  through  archived  files  to 
determine  which  items  recur  in  the  program  within  a 
specified  amount  of  time.  This  would  provide  the  history  of 
such  items  and  enable  managers  to  perform  additional 
analysis  to  determine  the  reasons  for  their  reoccurrence. 

Once  managers  are  aware  of  the  overall  condition  of  the 
CIP,  they  will  need  to  know  which  individual  items  are  in 
need  of  corrective  action.  Therefore,  several  managers 
recommended  that  the  ability  to  identify  the  individual 
items  having  defect  conditions  would  be  essential.  This 


129 


capability  is  not  explicitly  necessary  to  accomplish  the 
evaluation  of  the  CIP's  performance  using  percent  defects 
and  cycle  time;  however,  this  recommendation  is  a  logical 
next  step  in  providing  automated  tools  to  critical  item 
managers  for  direct  improvement  of  the  program. 

Guidance  must  be  established  identifying  at  which 
levels  CIP  performance  evaluation  will  be  accomplished. 
During  the  course  of  this  study,  managers  were  concerned  as 
to  what  level  and  which  organizations  the  measures  were  to 
be  applied.  This,  of  course,  must  be  negotiated  between 
operational  and  support  organizations;  however,  the  author 
of  this  study  recommends  that  measurement  capability  be 
designed  to  enable  use  of  these  measures  at  all  levels  and 
organizations  involved  with  CIP  management.  This  will 
enable  both  consistency  and  flexibility  of  measurement 
throughout  the  entire  program. 

To  ensure  the  proper  use  of  these  measures,  thorough 
education  of  all  managers  who  will  be  exposed  to  their 
products  must  also  be  accomplished.  These  measures,  percent 
defects  and  cycle  time,  are  designed  to  enhance  and 
contribute  to  continuous  improvement  of  the  process  they 
monitor.  Theoretically,  as  the  CIP's  performance  is 
improved,  more  attention  can  be  given  to  proactively  manage 
the  lower  two  hurdles  of  the  CIP  (problem  and  potential 
critical).  To  achieve  this  objective,  critical  item 
managers  at  all  levels  must  be  thoroughly  familiar  with  the 
purpose  and  meaning  of  the  measures  that  evaluate  its 
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performance.  Only  through  this  awareness  will  proper 
management  actions  be  recognized  and  implemented.  Without 
such  awareness,  misinformed  managers  will  likely  waste 
resources  as  well  as  degrade  the  program's  improvement  of 
weapon  system  availability. 

Chapter  V  Summary 

This  chapter  has  stated  the  conclusions  of  this  study 
and  provided  several  recommendations  for  pursuing  the  use  of 
percent  defects  and  cycle  time  in  measurement  of  the  CIP's 
performance.  As  a  result  of  researching  investigative  and 
measurement  questions,  this  study  has  determined  that 
percent  defects  and  cycle  time  could  be  useful  for  the 
effective  management  of  the  Air  Force  CIP.  However,  only 
through  further  research  and  implementation  of  these  two 
measures  will  direct  results  be  gained.  To  address  the 
further  research  and  development  of  these  measures,  several 
recommendations  were  provided.  Included  in  those 
recommendations  was  the  implementat ion  of  a  formal  test  of 
these  measures  in  a  field  environment,  establishment  of 
guiding  cycle  time  objectives,  automation  of  the  measurement 
process  in  conjunction  with  WSMIS,  development  of  an 
automated  capability  for  managers  to  identify  individual 
items  possessing  defect  conditions,  and  the  education  of 
managers  who  will  use  these  measures  to  improve  the  program. 
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The  author  of  this  study  believes  that  the  need  for  a 
CIP  evaluation  capability  was  clearly  established.  For  only 
through  meaningful  measurement  of  a  system's  performance  can 
management  determine  which  actions  are  necessary  to  promote 
continuous  improvement. 
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ACIN 

AFAA 

AFLC 

AFLCR 

AFLMC 

AFM 

ALC 

AWP 

CLS 

CIM 

CIP 

DOD 

ES 

FAD 

GWAM 

IMS 

LCL 

MAJCOM 
MDS 
MI  CAP 
MM 

NMCB 

NMCM 

NMCS 

NORS 


Appendix  A:  Conunonly  Used  Acronyms 

Automated  Critical  Item  Network 

Air  Force  Audit  Agency 

Air  Force  Logistics  Command 

Air  Force  Logistics  Command  Regulation 

Air  Force  Logistics  Management  Center 

Air  Force  Manual 

Air  Logistics  Center 

Awaiting  Parts 

Contractor  Logistics  Support 

Critical  Item  Manager 

Critical  Item  Program 

Department  of  Defense 

Equipment  Specialist 

Force  Activity  Designator 

Get  Well  Assessment  Module 

Item  Management  Specialist 

Lower  Control  Limit 

Major  Command 

Mission  Design  Series 

Mission  Capable 

Materiel  Manager 

Not  Mission  Capable  Both 

Not  Mission  Capable  Maintenance 

Not  Mission  Capable  Supply 

Not  Operationally  Ready  Supply 
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PMS 


Production  Management  Specialist 


POS 

Peacetime  Operating  Stock 

RAM 

Readiness  Assessment  Module 

REALM 

Requirements  Execution  Availability 
Module 

Logistics 

SAC 

Strategic  Air  Command 

SAFE 

Supportabi 1 ity  Analysis  Forecasting 
Evaluation  System 

and 

SAM 

Sustainability  Assessment  Module 

SPM 

System  Program  Manager 

SQC 

Statistical  Quality  Control 

SRD 

Standard  Reporting  Designator 

TAG 

Tactical  Air  Command 

TQM 

Total  Quality  Management 

UCL 

Upper  Control  Limit 

UMMIPS 

Uniform  Materiel  Movement  and  Issue 
System 

Priority 

WSF 

Weapon  System  Factor 

WSMIS 

Weapon  System  Management  Information 

System 
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endix  B:  Information  Package  for  Critical  Item  Managers 


PERCENT  DEFECTS  AND  CYCLE  TIME 
Two  Potential  Measures  for  Evaluation  of  the  CIP 

Background 

The  Air  Force  Critical  Item  Program  (CIP)  has  been  an 
official  management  concern  of  both  operational  and 
supporting  commands  since  the  height  of  the  Vietnam  War 
(9:1).  This  valid  concern  is  due  to  the  strong  impact  that 
scarce  parts  and  equipment  can  have  on  weapon  system 
availability,  and  ultimately  on  the  successful  outcome  of 
operational  missions.  Over  the  last  two  and  a  half  decades, 
numerous  studies,  selection  criteria,  and  administrative 
systems  have  been  developed  for  this  program  in  hopes  of 
improving  weapon  system  availability  (1;  2;  5;  8;  9;  10). 
Though  incremental  improvements  have  been  noted,  substantial 
disagreements  still  exist  between  operational  and  supporting 
commands  concerning  management  of  the  CIP  (6).  This 
disagreement  has  slowed  improvements  to  the  program  in  the 
past  and  continues  to  do  so  today. 

A  major  shortfall  of  the  current  program  (identified  in 
a  recent  Air  Force  Audit  Agency  study  and  recognized  by  CIP 
officials)  is  a  means  of  evaluating  the  CIPs  performance  (3; 
6).  Because  of  this  shortfall,  common  goals  and  management 
objectives  are  difficult  to  communicate  between  operational 
and  supporting  organizations.  Establishment  of  common 
measures  for  CIP  performance  could  improve  communication 
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among  commands  and  thereby  improve  attainment  of  stated 
goals  and  objectives.  Continuation  of  the  program  without 
establishment  of  a  performance  evaluation  capability  is  most 
likely  to  retain  the  functional  differences  now  occurring 
between  operational  and  supporting  commands. 

To  address  this  issue,  an  AFIT  Thesis  has  been  directed 
at  analyzing  possible  measures  for  evaluating  CIP 
performance.  Through  review  of  measurement  practices  of 
both  private  industry  and  government  organizations,  two 
measures,  percent  defects  and  cycle  time,  have  been 
identified  as  potentially  beneficial  to  the  CIP  (4;  7;  11). 
These  measures  used  together,  could  provide  a  means  for 
increasing  communication  of  common  goals  and  objectives 
among  operational  and  supporting  commands .  •  The  remainder  of 
this  report  will  therefore  concentrate  on  a  brief 
description  of  how  these  measures  can  be  applied  to  the  CIP. 

CIP  Pitfalls 

In  determining  which  measures  to  analyze,  an  in-depth 
study  was  accomplished  addressing  the  requirements  of  the 
CIP.  From  this  study,  several  pitfalls  currently  occurring 
in  the  program  were  identified.  These  pitfalls  are  listed 
below: 

1.  Items  selected  into  the  CIP  are  not  identified  by 
the  Weapon  System  Management  Information  System 
(WSMIS)/Get  Well  Assessment  Module  (GWAM. 

2.  Items  selected  into  the  CIP  do  not  meet 
preestablished  MAJCOM  criteria  in  accordance  with  AFM 
67-1  . 
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3.  Items  previously  having  been  deleted  from  the  CIP 

reenter  the  program  due  to  recurring  problems. 

Each  of  these  conditions  can  be  described  as  a  defect 
of  the  program.  For  the  purposes  of  this  study,  a  defect  is 
defined  as  an  item  fulfilling  a  prescribed  defect  condition 
(one  of  the  three  pitfalls  stated  above).  Every  item 
entering  or  maintained  in  the  CIP  has  the  potential  of 
possessing  one  or  all  of  these  defects.  These  defects  are 
attributes  in  nature,  in  that  CIP  items  either  fulfill  or  do 
not  fulfill  the  defect  condition.  Subsequently,  each  item 
in  the  CIP  can  also  be  measured  for  each  of  these 
conditions.  The  goal  of  such  meas'.'rement  is  to  track  the 
occurrence  of  each  condition  in  order  to  reduce  future 
defects  through  appropriate  management  action.  As  defects 
are  reduced,  the  performance  and  effectiveness  of  the  CIP 
will  improve.  Alternately,  if  defects  increase,  recognition 
of  this  increase  i.s  immediate,  also  enabling  appropriate 
management  action.  The  obvious  goal  for  all  organizations 
involved  in  CIP  manageni«^nt  would  be  to  reduce  the  number  of 
defects  to  an  absolute  minimum.  Agreement  on  this  common 
goal  between  operational  and  supporting  organizations  would 
facilitate  negotiation  of  specific  management  actions  and 
criteria  necessary  to  realize  their  goal.  In  short,  this 
would  facilitate  removal  of  functional  barriers  and  promote 
a  more  unified  approach  to  managing  critical  items. 

Another  pitfall  identified  was  the  time  necessary  to 
process  an  item  through  the  CIP.  This  time  includes 
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identification  of  the  item,  developmp”t  of  a  get-well  plan, 
approval  of  the  get-well  plan,  implementation  of  actions 
specified  by  the  plan,  and  fnally  deletion  of  the  item  from 
the  CIP  once  it  is  considered  well.  Though  the  CIP  is 
designed  to  expedite  acquisition  and/or  repair  of  critical 
items,  variation  in  item  cycle  time  ranges  from  one  month  to 
upwards  of  two  years.  The  upper  bound  of  this  variation 
strongly  suggests  there  is  the  possibility  for  improvement. 
Reduction  of  cycle  time  would  improve  weapon  system 
availability  and  allow  limited  resources  to  be  applied  to 
other  problem  items  more  quickly.  Assuming  continuous 
improvement  (as  management  of  the  system  is  designed  to  do), 
resources  would  eventually  be  available  for  more 
preventative  actions  than  are  currently  possible.  Concern 
for  cycle  time  is  also  important  to  ensure  management  does 
not  take  actions  which  might  reduce  defects  while 
simultaneously  increasing  cycle  time.  Private  industry 
experience  has  proven  that  the  use  of  these  two  measures 
together  can  produce  vast  improvements  in  process  quality 
(  / ;  11). 

Using  t.he  Measures  of  Percent  Defects  and  Cycle  Time 

Before  describing  the  measures  of  percent  defects  and 
cycle  time,  the  reader  is  encouraged  to  review  Attachments  1 
and  2  so  as  to  be  generally  familiar  with  their  structure 
and  content.  Included  in  these  attachments  are  instructions 
on  how  each  chart  is  developed. 
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One  of  the  major  benefits  of  using  these  two  measures 
is  that  they  can  be  used  at  all  levels  of  an  organization 
easily  and  with  clear  understanding  (11).  In  turn, 
application  of  these  measures  to  the  Air  Force  CIP  can 
effectively  be  used  starting  at  the  unit  level  all  the  way 
up  through  the  Air  Force  level.  This  is  a  powerful 
characteristic  due  to  the  application  of  the  same  language 
and  measurements,  thus  enabling  all  levels  of  management  to 
strive  for  the  same  goals  and  objectives.  To  apply  these 
measurements,  two  well  established  control  charts  are  used. 

P  charts  (for  percent  defectives)  are  used  to  track  defects, 
and  xR  charts  (also  called  average  range  charts  due  to  their 
tracking  of  average  and  variation  values  within  a  process) 
are  used  to  track  cycle  time.  Each  of  these  charts  is 
described  below. 

P  Chart.  As  mentioned  above,  the  P  chart  is  used  to 
track  defects.  As  the  CIP  is  updated  each  quarter,  defects 
would  be  calculated  quarterly  too.  To  accomplish  this,  each 
defect  condition  is  evaluated  for  each  item  and  a  sum  total 
for  each  condition  annotated  in  the  appropriate  column  of 
the  P  chart  (see  Attachment  1,  Lines  2-4).  Once  all  items 
have  been  evaluated  for  defect  conditions,  a  grand  total  of 
all  defects  is  calculated  (see  Attachment  1,  Line  5).  This 
number  is  then  divided  by  the  total  number  of  CIP  items 
reviewed  to  determine  the  percent  defects  measure  (see 
Attachment  1,  Lines  5-7).  The  percent  defects  measure  is 
then  carried  to  Line  1  of  the  chart  and  graphically 
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annotated.  The  upper  control  limit  (UCL)  is  a 
preestablished  guideline  to  alert  management  when  the 
process  is  in  or  out  of  control.  The  UCL  is  calculated 
using  the  20  most  recent  data  points  and  the  appropriate 
formula  identified  in  Attachments  1  and  2.  Because  data  are 
not  currently  maintained,  20  historic  data  points  were  not 
available.  Therefore,  UCLs  shown  in  the  three  attachments 
are  hypothetical.  Any  measures  annotated  above  the  UCL 
signifies  that  management  action  is  necessary  to  explain 
and/or  correct  the  out  of  control  situation.  Measurement  of 
defects  using  this  chart  identifies  to  management  how  well 
the  CIP  is  performing  for  the  overall  area  being  measured, 
and  breaks  down  the  CIPs  performance  within  each  defect 
condition.  This  provides  management  with  significant 
information  necessary  to  make  appropriate  corrective 
actions . 

xR  Chart.  The  xR  chart  is  used  to  track  cycle  time. 
Because  items  are  deleted  from  the  CIP  every  month,  this 
chart  can  be  updated  monthly.  xR  charts  work  best  when  at 
least  five  equal  sample  areas  are  measured  together.  In  the 
Air  Force  for  example,  this  could  be  Mission  Design  Series 
(MDS),  and/or  Air  Logistics  Centers  (ALC),  and/or  Major 
Commands  (MAJCOM).  To  update  the  chart,  the  average  cycle 
time  for  the  selected  areas  is  totaled  to  establish  a  summed 
value  (see  Attachment  2,  Lines  1-6).  This  value  is  then 
divided  by  the  number  of  entries  to  calculate  an  average 
value  (see  Attachment  2,  Line  7).  Next,  the  range  over 
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which  the  cycle  time  occurred  among  the  selected  areas  is 
determined.  This  value  is  the  largest  cycle  time  measure 
minus  the  smallest  cycle  time  measure  within  the  selected 
areas  for  that  month  (see  Attachment  2,  Line  8).  Once  the 
average  and  range  are  calculated,  the  results  are 
graphically  annotated.  For  cycle  time  to  be  in  control, 
both  the  average  and  range  results  must  fall  below  the  UCL 
boundary  lines.  If  either  UCL  boundary  line  is  breached, 
then  management  should  take  action  to  explain  or  correct  the 
situation.  Measurement  of  cycle  time  by  this  means  ensures 
that  offsetting  values  do  not  communicate  a  false  measure  of 
process  control . 

Continuous  Improvement 

Several  of  the  management  benefits  attainable  through 
these  measures  have  already  been  described,  including: 
better  organizational  communication,  better  defined  goals 
and  objectives,  and  specific  identification  of  problem 
areas.  These,  however,  are  not  the  only  beneficial 
applications  for  percent  defects  and  cycle  , ime.  The  basic 
premise  of  these  measures  is  to  instill  the  process  of 
continuous  improvement.  This  is  accomplished  by  continually 
shrinking  the  variation  in  defects  and  cycle  time  once  the 
system  has  been  brought  under  control.  As  described 
earlier,  all  measures  must  fall  under  the  upper  control 
limit  in  order  to  be  in  control.  The  continuous  process  of 
bringing  the  system  under  control  and  then  shrinking  down 


141 


the  UCL  will  eventually  lead  to  a  system  operating  at  near 
optimal  performance.  Attachment  3  provides  a  full 
application  of  the  P  and  xR  charts  for  five  SAC  aircraft 
including:  B-1,  B-52,  C-135,  E-4,  and  F-111  aircraft.  The 
purpose  of  this  attachment  is  to  allow  the  reader  to  view 
one  possible  application  of  these  measures  using 
comparatively  realistic  data.  As  indicated  by  these  charts, 
the  current  CIP  is  far  from  being  in  control.  A  cumulative 
average  of  percent  defect  rates  shows  that  measures 
currently  range  from  43  to  65  percent.  This  is  far  above 
the  hypothetical  UCL  level  of  30  percent.  This  indicates  to 
management  that  action  must  be  taken  to  bring  defects  back 
to  a  level  where  they  are  in  control.  Cycle  time  averages 
between  6.5  to  7.0  months  with  variability  of  6.4  to  8.1 
months.  These  figures  are  below  the  possible  UCL  of  12 
months  indicating  that  the  system  is  in  control  and 
therefore,  a  lower  UCL  can  be  established  to  strive  for 
better  performance. 

Data  Limitations.  Due  to  the  nonavailability  of 
complete  historic  information,  portions  of  the  charts  in 
Attachment  3  could  not  be  completed  with  full  or  in  some 
circumstances  actual  data.  Data  contained  in  the  P  charts 
have  full  data  for  the  months  of  June  and  April.  Data 
reported  for  the  months  of  September  and  January  are  for 
items  managed  at  Warner -Robbins  ALC  only.  In  addition,  in 
evaluation  of  the  three  defect  conditions,  #  NOT  GWAM 
IDENTIFIED  was  evaluated  for  all  months,  #  RECURRING  ITEM 
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was  evaluated  for  the  month  of  April  only,  and  #  NOT  MEET 
CRITERIA  was  not  evaluated  at  all  due  to  time  limitations  to 
collect  such  data.  Addressing  cycle  time,  four  data  points 
were  deemed  necessary  to  demonstrate  the  usefulness  of  the 
xR  chart;  however,  only  data  from  May  and  June  were 
available.  To  ensure  that  the  usefulness  of  this  chart  was 
properly  communicated,  data  were  arbitrarily  extrapolated 
for  all  SAC  aircraft  for  the  months  of  March  and  April.  All 
other  data  contained  in  these  charts  are  actual. 

Conclusion 

Once  again,  the  purpose  of  this  presentation  is  to 
introduce  two  possible  measures  for  evaluation  of  the  CIP. 

It  is  directed  at  field  experts,  who  will  be  asked  to 
provide  their  opinions  regarding  ' he  potential  and 
practicality  of  these  measures  as  applied  to  the  continuous 
improvement  of  the  CIP.  Where  possible,  full  and  actual 
data  were  used  in  presentation  of  these  two  measures.  Where 
data  were  not  available,  subsets  or  extrapolated  data  were 
presented  as  stated  above.  Readers  are  urged  to  review  this 
report  thoroughly  in  order  to  fully  understand  the 
application  of  the  measures  percent  defects  and  cycle  time 
presented  in  this  report. 
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PERCENT  DEFECTS 
P  CONTROL  CHART 

MAJCOM:  _ 
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#  REVIEWED 


7. 

%  DEFECTS 


LEGEND:  »  NOT  GWAM  IDENTIFIED 
»  NOT  MEET  CRITERIA 

#  RECURRING  ITEM 
%  DEFECTS 


CIP  item  not  identified  by  GWAM 
CIP  item  not  fulfilling  MAJCOM 
cr  iter ia 

CIP  item  previously  archived 
(Sum  Defects)  /  ( #  Reviewed) 


Page  1  of  2 


Attachment 


144 


P  Chart  Completion  Instructions 


Line 


Instructions 


1  This  value  is  carried  from  line  1,  once  all 
calculations  have  been  completed.  Data  is  then 
graphically  annotated  in  the  chart  above  line  1. 

2  Items  selected  to  the  CIP  as  critical  are  compared 
to  those  items  identified  by  the  WSMIS/GWAM 
Candidate  List  for  all  three  hurdles.  Items  in 
the  CIP  which  are  not  identified  in  the  WSMIS/GWAM 
Candidate  List  are  considered  a  defect.  The  total 
of  these  defects  is  entered  on  line  2. 

3  Items  selected  to  the  CIP  are  reviewed  to 
determine  if  they  meet  current  MAJCOM  criteria. 
Items  failing  to  meet  criteria  are  considered  a 
defect.  The  total  of  these  defects  is  entered  on 
line  3.  Automated  capability  to  perform  this 
review  is  currently  not  available. 

4  Items  leaving  and  reentering  the  CIP  on  a 
recurring  basis  are  considered  defects.  The  total 
of  these  defects  is  entered  on  line  4.  This  value 
can  be  determined  by  reviewing  archived  Form  74s. 

5  This  is  the  sum  total  of  lines  2-4. 

6  This  is  the  total  number  of  items  reviewed  for 
defect  conditions 

7  Percent  defects  is  calculated  by  dividing  the 
value  of  line  5  by  the  value  of  line  6. 


P  Chart  Upper  Control  Limit  Formula; 


crcxs 


'P+3J 


p(i-p) 


n 


(1) 


where : 

UCL  =  upper  control  limit 
Pt>«c^  =  average  number  of  defects 
n  =  sample  size 

NOTE:  UCL  values  used  in  this  study's  charts  are  hypothetical  due 
to  nonavailability  of  data. 
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CYCLE  TIME 

AVERAGE/RANGE  CONTROL  CHART 

MAJCOM:  _ 

YEAR:  _ 


x:  o  z  6h  X  CO 


xR  Chart  Completion  Instructions 

Line _ Instructions _ 

1-5  This  is  the  average  cycle  time  for  each 

identified  MDS  as  determined  from  data  extracted 
from  the  monthly  WSMIS/GWAM  Add/Delete  Reports. 

6  This  is  the  sum  of  the  averages  for  all  MDSs  for 
the  specified  month. 

7  This  is  the  overall  average  cycle  time 
calculated  by  dividing  line  6  by  the  number  of 
measured  MDSs  for  the  specified  month. 

8  This  is  the  range  over  which  individual  MDS 
averages  occurred.  This  value  is  calculated  by 
subtracting  the  smallest  MDS  cycle  time  average 
from  the  largest  MDS  cycle  time  average  in  each 
month.  Average  and  range  values  are  then 
graphically  annotated  in  the  appropriate  charts 
below  line  8. 


Average  Upper  Control  Limit 

UCI^mX^A^H  (  2 ) 


where : 

UCLxbas  ~  upper  control  limit  for  Xb«c 

Xb«E  =  average  cycle  time  of  individual  weapon  systems 
Xb.KbsE  =  average  of  the  average  weapon  system  cycle  time 
Rb««  =  average  range  over  which  sample  data  are  spread 
Aa  =  special  factor  for  control  limit 

Range  Upper  Control  Limit 

UCLjt^Djt  ( 3 ) 

where : 

UCLb  =  upper  control  limit  for  range 

RbbE  =  average  range  over  which  sample  data  are  spread 
D<  =  special  factor  for  upper  control  limit 

NOTE:  UCL  values  used  in  this  study's  charts  are  hypothetical 
due  to  nonavailability  of  data. 
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PERCENT  DEFECTS 
P  CONTROL  CHART 
MAJCOM:  SAC 

Aircraft:  C-135 
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Appendix  C:  Questions  Critical  Item  Managers  Were  Asked 


1.  Do  you  believe  any  benefits  will  be  gained  by  using 
percent  defects  to  measure  the  CIP  performance?  If  yes, 
what  are  they? 


2.  Do  you  believe  any  disadvantages  will  result  by  using 
percent  defects  to  measure  the  CIP's  performance?  If  yes, 
what  are  they? 


3.  Do  you  believe  any  benefits  will  be  gained  by  using 
cycle  time  to  measure  the  CIP  performance?  If  yes,  what  are 
they? 


4.  Do  you  believe  any  disadvantages  will  result  from  using 
the  measure  of  cycle  time  to  measure  the  CIP's  performance? 
If  yes,  what  are  they? 


5.  How  is  the  CIP  being  measured  now? 


6.  What  are  the  benefits  gained  by  the  measures  currently 
used  to  manage  the  CIP? 


1.  What  are  the  disadvantages  realized  by  the  measures 
currently  used  to  manage  the  CIP? 


8.  Do  these  current  measures  produce  effective  management 
of  the  CIP? 


9.  Having  been  presented  and  considered  the  measures  of 
percent  defects  and  cycle  time,  would  you  recommend  these 
measures  be  used?  Why,  or  why  not? 


156 


Bibliography 

1.  Adams,  Col  Gerald  C.  and  Harry  B.  Yospe.  The  Economics 
of  National  Security.  Washington  DC:  Industrial 
College  of  the  Armed  Forces,  1965. 

2.  Agnor,  Capt  Raymond  M.  Jr.  and  Capt  Douglas  C.  Topping. 
An  Analysis  of  the  MORS  Hour  Selection  Criterion  for 
Aircraft  Critical  Items.  MS  thesis,  AFIT/SLSR-31-71A. 
School  of  Systems  and  Logistics,  Air  Force  Institute  of 
Technology  (AU),  Wr ight-Patterson  AFB  OH,  February  1971 
( AD877524 ) . 

3.  Air  Force  Audit  Agency.  Follow-up  Audit  --  Review  of 
the  Air  Force  Special  Management  Program  for  Critical 
Supply  Items.  Project  5126127.  Wr ight-Patterson  AFB 
OH,  September  30,  1986. 

4.  Air  Force  Audit  Agency.  Management  of  High 
NORS/Cr i t ical  Repair  Cycle  Assets.  Audit  5314-31. 

Kelly  AFB  TX,  20  May  1969. 

5.  Air  Force  Audit  Agency.  Review  of  the  Air  Force 
Special  Management  Program  for  Critical  Supply  Items. 
Project  807713,  Wr ight-Patterson  AFB  OH,  5  May  1982. 

6.  Air  Force  Logistics  Command.  Logistics  Management 
Systems .  Information  System  Brochure.  LMSC/SZ, 

Wr  ight-Patterson  AFB  OH,  24  February  1989. 

7.  Air  Force  Logistics  Management  Center.  Critical  Item 
Management  Criteria.  AFLMC  Project  LS841129.  Gunter 
AFS:  June  1986. 

8.  Amsden,  Robert  T.  and  others.  SPC  Simplified; 

Practical  Steps  to  Quality.  White  Plains:  Kraus 
International  Publications,  1986. 

9.  Brassard,  Michael.  The  Memory  Jogger.  Methuen: 
GOAL/QPC,  1988. 

10.  Chase,  Richard  B.  and  Nicholas  J.  Aquilano.  Product  ion 
and  Operations  Management:  A  Life  Cycle  Approach  (Fifth 
Edition).  Boston:  Irwin,  1989. 

11,  Condojani,  Capt  Stanley  S.  and  Capt  Warren  P.  Welsh. 

An  Analysis  of  Critical  Items  in  Cargo  Aircraft.  MS 
thesis.  AFIT/SLSR-24-67 ,  School  of  Systems  and 
Logistics,  Air  Force  Institute  of  Technology  ( AU ) , 

Wr  ight-Patterson  AFB  OH,  August  1967  (AD  824908). 


157 


12.  Correa,  Luis,  Audit  Manager.  Personal  Interview.  Air 
Force  Audit  Agency,  AFAA/QLS,  Wr ight-Patter son  AFB  OH, 
24  January  1991. 

13.  Deming,  W.E.  "On  Some  Statistical  Aids  Towards  Economic 
Production,"  Interfaces .  5:  1-15  (August  1975). 

Deming,  W.  Edwards.  Out  of  the  Crisis.  Cambridge: 
Massachusetts  Institute  of  Technology,  1982. 

Department  of  the  Air  Force.  USAF  Supply  Manual:  Air 
Force  Critical  Item  Program.  AFM  67-1,  Volume  I,  Part 
One,  Chapter  26.  Washington:  HQ  USAF,  30  April  1990. 

Emmelhainz,  Lt  Col  Larry  W.  "TQM  Principles  and 
Measures:  Key  to  Successful  Implementation."  Air  Force 
Journal  of  Logistics  (in  press).  1^:  34-36  (Summer 
1991 )  . 

17.  Emmelhainz,  Lt  Col  Larry  W. ,  Director  of  Research  and 
Consulting.  Personal  Interview,  Air  Force  Institute  of 
Technology,  School  of  Systems  and  Logistics,  Wright- 
Patterson  AFB  OH,  23  January  1991. 

18.  Emory,  C.  William.  Business  Research  Methods  (3rd 
Edition).  Homewood:  Irwin,  1985. 

19.  Fine,  Charles  H.  and  Evan  L.  Porteus.  "Dynamic  Process 
Improvement,"  Operations  Research.  37 :  580-591  (July- 
August  19  89  )  . 


20.  Freedman,  Lawrence.  "Logistics  and  Mobility  in  Modern 
Warfare,"  Armed  Forces.  5.:  65-70  (February  1986). 

21.  Fromm,  Maj  Roger,  Program  Manager .  Personal  Interview. 
WR-ALC/MMH,  Robins  AFB  GA,  26  September  1990. 

22.  Get  Well  Assessment  Module.  Functional  Description 
D087E.  Contract  F33600-89-C-0100,  CDRL  C006,  Dynamics 
Research  Corporation,  30  September  1990. 

23.  Gilmore,  Harold  L.  "Continuous  Incremental 
Improvement:  An  Operations  Strategy  for  Higher  Quality, 
Lower  Costs,  and  Global  Competitiveness,"  SAM  Advanced 
Management  Journal.  55 :  21-25  (Winter  1990). 

24.  Grant,  Eugene  L.  and  Richard  S.  Leavenworth. 

Statistical  Quality  Control  (Sixth  Edition).  New  York: 
McGraw  Hill,  1988. 

25.  Greiner,  Anne  Claire.  "Quality  Counts,"  Technology 
Review.  9  3 :  19-20  (July  1990). 


26.  Hansen,  General  A.G.  "Total  Quality  Management:  A 
Powerful  Solution  to  the  Logistics  Challenge,"  Program 
Manager ,  28 :  9-12  (January  -  February  1989). 

27.  Harrington,  Dr.  H.J.  The  Improvement  Process.  New 

York:  McGraw-Hill  Book  Company,  1987. 

28.  Harrington,  James.  Excellence  the  IBM  Wav.  Milwaukee: 
ASQC  Quality  Press,  1988. 

29.  Headquarters  Air  Force  Logistics  Command.  AFLC  Surge 
Contingency  Plan.  AFLC  Plan  70.  Wr ight-Patterson  AFB 
OH,  21  October  1988. 

30.  Herge,  Lt  Col  Donna  C.,  Statistics  Branch  Chief, 
Department  of  Math  Statistics.  Personal  Interview. 

Air  Force  Institute  of  Technology,  School  of 
Engineering,  Wr ight-Patterson  AFB  OH,  24  April  1991. 

31.  Ishikawa,  Dr.  Kaoru.  Guide  to  Quality  Control.  White 

Plains:  Asian  Productivity  Organization,  1982. 

32.  Juran,  J.M.  Juran  on  Planning  for  Quality.  New  York: 
The  Free  Press,  1988. 

33.  Juran,  J.M.  and  Frank  M.  Gyrna,  Jr.  Quality  Planning 
and  Analysis  (2nd  Edition).  New  York:  McGraw-Hill 
Book  Company,  1930. 

34.  Kerlinger,  Fred  N.  Foundations  of  Behavioral  Research 
(Second  Edition).  New  York:  Holt,  Rinehart  and  Winston 
Inc.,  1973. 

35.  Kume,  Hitoshi.  Statistical  Methods  for  Quality 
Improvement .  Tokyo:  3A  Corporation,  1985. 

36.  Lynch,  Christopher  R.,  Logistics  Management  Specialist 
Critical  Item  Program.  Telephone  Interview.  HQ 
AFLC/MMIM,  Wr ight-Patterson  AFB  OH,  19  February  1991. 

37.  Main,  Jeremy.  "How  to  Win  the  Baldrige  Award," 

Fortune .  121;  101+  (23  April  1990). 

38.  McClave,  James  T.  and  P.  George  Benson.  Statistics  for 
Business  and  Economics  (Fourth  Edition).  London; 
Collier  MacMillan  Publishers,  1988. 

39.  Messina,  William  S.  Statistical  Quality  Control  for 
Manufacturing  Managers.  New  York:  John  Wiley  &  Sons, 
1987  . 


159 


40. 


Norton,  Capt  Philip  F.  and  Capt  Arthur  V.  Tennyson.  An 
Examination  o£  Certain  Causes  of  Critical  Supply  Status 
to  Develop  a  Method  to  Forecast  Potential  Item 
Criticality.  MS  thesis,  AFIT/SLSR-23-67 .  School  of 
Systems  and  Logistics,  Air  Force  Institute  of 
Technology  (AU),  Wr ight-Patter son  AFB  OH,  August  1967 
(AD  825296 )  . 

41.  Ogan,  Maj  Andrew  J.  "Aircraft  Availability:  A  Concept 
Whose  Time  Has  Come!"  Air  Force  Journal  of  Logistics. 

1 2 1  36-38  (Spring  1988  ). 

42.  Peterson,  Lt  Col  Tim  0.  "Performance  Support  Tools: 
Part  of  the  Quality  Work  Force  Equation,"  Air  Force 
Journal  of  Logistics.  1 4 :  18-21  (Winter  1990). 

43.  Phillips,  Florence  L.  and  2Lt  Floyd  A.  Hooks.  An 
Evaluation  of  the  NORS  Selection  Criteria  for  the  High 
NQRS  Critical  Items  Program.  MS  thesis,  AFLCR  67-21, 
School  of  Systems  and  Logistics,  Air  Force  Institute  of 
Technology  (AU),  Wr ight-Patterson  AFB  OH,  August  1969 
(AD-863846  )  , 

44.  Ramroth,  Col  Albert  M.  "Adding  CLOUT  to  Logistics," 

Air  Force  Journal  of  Logistics.  11 :  1-3  (Fall  1987), 

45.  Readiness  Assessment  Module.  Functional  Description 
DC87B/D.  Contract  F33606-85-G-0058/0005,  CDRL  ABOl', 
Dynamics  Research  Corporation,  15  August  1988, 

46.  Reynolds,  Larry,  "Promoting  Quality  in  the  Public  and 
Private  Sectors,"  Management  Review.  7 8 :  16-17  (May 
1939  )  . 

47.  Scherkenbach,  William  W.  The  Deminq  Route  to  Quality 
and  Productivity.  Washington  DC:  CEEPress  Books, 

1988  . 

43.  Seifert,  Kenneth  E,  and  Capt  Martin  W.  Nakunz.  An 
Analysis  of  USAF  Stock  Control  and  Requisitioning 
Policies  and  Their  Effect  on  the  Management  of  Critical 
I  terns .  MS  thesis,  AFIT/SLR-28-67 .  School  of  Systems 
and  Logistics,  Air  Force  Institute  of  Technology  ( AU ) , 
Wr ight-Patterson  AFB  OH,  August  1967  f AD  823943). 

49.  Sink,  D.  Scott  and  Thomas  C.  Tuttle.  Planning  and 

Measurement  in  You  Organization  of  the  Future .  New 
York:  Institute  of  Industrial  Engineers,  1939. 

50.  Smith,  William,  Vice  President  and  Senior  Quality 
Assurance  Manager,  Motorola  Commun icat ions  Sector. 
"Total  Customer  Satisfaction."  Speech,  Wright- 
Patterson  AFB  OH,  January  1991. 


160 


51.  Statistix  3.1.  St  Paul:  Analytical  Software,  1989. 

52.  Talley,  Debi,  Critical  Item  Manager.  Telephone 
Interview.  OC-ALC/FMRRD,  Tinker  AFB  OK,  19  February 
1991  . 

53.  Thomson,  David  A.  Analysis  of  the  Critical  Item 
Performance  of  the  Gardner -Dannenbr ina  Aggregate 
Inventory  Model.  MS  thesis,  AFIT/GLM/LSM/855-78 . 
School  of  System  of  Logistics,  Air  Force  Institute  of 
Technology  (AU),  Wr ight-Patterson  AFB  OH,  September 
1985  (AD  A161785 )  . 

54.  Villarreal,  Robert,  Critical  Item  Manager.  Telephone 
Interview.  SA-ALC/FMRP-5,  Kelly  AFB  TX,  19  February 
1991  . 

55.  Walton,  Mary.  The  Demina  Management  Method.  New  York 
Perigee  Book,  1986. 


161 


Vita 


Captain  Robert  R.  Lee  was  born  on  6  March  1964  in 
Oxnard  California.  He  graduated  from  Mesa  High  School  in 
Mesa,  Arizona,  in  1982.  In  1986,  he  graduated  from  Arizona 
State  University  in  Tempe,  Arizona  with  a  Bachelor  of 
Science  in  Management.  Immediately  following  graduation  he 
was  commissioned  a  second  lieutenant  in  the  United  States 
Air  Force.  He  has  since  been  assigned  to  Eglin  AFB,  Florida 
where  he  served  in  various  management  positions  including 
base  level  and  acquisition  logistics.  During  his  assignment 
at  Eglin  AFB,  he  graduated  from  Troy  State  University  in 
Troy,  Alabama,  with  a  Master  of  Science  in  Management.  He 
entered  the  School  of  Systems  and  Logistics,  Air  Force 
Institute  of  Technology,  in  May  1990.  Captain  Lee’s  next 
assignment  is  to  Warner  Robins  Air  Logistic  Center,  Robins 
AFB,  Georgia. 

Permanent  address:  1619  Lookout  Drive 

Coeur  d'  Alene,  Idaho  83814 


162 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
0MB  No  0704  01 88 


_,oi'C  ■'POO'’*  ->  •;  ou' JP''  *  '"'S  s  '  ■'  ur  3»>r  '.'*^oop^e  r-ri'-ai^q  tpp  '<zr  ro', ipw.p.j  instr^-.t.  :rs  .‘^^r  "t  '• 

}  jtPPfiPq  iP<3  ''’POJUl  PPPOPO.  iPfl  •.CP'O'elir'q  »pO  •"•p  .  Ill**'  TiOn  .■,r  .-'t  ■'r'»*dticn  )»»na  •;  jmmePts  rpq  jrainq  th;s  Duf  3Pn  UP  '.f 

OiiPt'iCP  t  :v.  P  Pk  -.aiP  4  OP'j  •  ?f  '’•‘Cuv  ir  •;  'Pis  Ourqpp  /<  i^Pir.iton  ■^<*4<3au4P*P'S -;fPCT^<d»fP  ODP‘’-^t'''P'>  i^a 

-'.r  "j;.,-''  .  •’ *Cy  >ra*-  V*  •*  .*  M  i-- t'-a  ^'-c  jp!  ioe''<vrf«  f^pou- t-O'' ",  A'nnru'p 


1.  AGENCY  USE  ONLY  [Leave  Olank) 


2  REPORT  DATE 
September  1991 


3  REPORT  TYPE  AND  DATES  COVERED 


4.  TITLE  AND  SUBTITLE 

AN  ANALYSIS  OF  MEASURES  USED  TO  EVALUATE  THE  AIR  FORCE 
CRITICAL  ITEM  PROGRAM 


6.  AUTHOR(S) 


5.  FUNDING  NUMBERS 


Robert  R.  Lee,  Captain,  USAF 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Institute  of  Technology,  WPAFB  OH  45433-6583 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

AFIT/GLM/LS C/9 IS - 4 0 

9.  SPONSORING  MONITORING  AGENCY  NAME(S)  AND  AOOflESS(ES) 

10.  SPONSORING  MONITORING 

AGENCY  REPORT  NUMBER 

11.  SUPPLEMENTARY  NOTES 


12a.  DISTRIBUTION  AVAILABILITY  STATEMENT 


12b.  distribution  code 


Aporoved  for  public  release;  distribution  unlimited 


13.  ABSTRACT  (Mammum  200  words) 

The  purpose  of  this  study  was  to  evaluate  the  potential  of  two  measures,  percent 
defects  and  cycle  time,  to  measure  the  overall  performance  of  the  Air  Force  Critical 
Item  Program  (CIP) .  The  need  for  such  study  was  generated  from  the  current  absence 
of  any  measurement  system  available  to  evaluate  the  CIP.  This  research  used  these 
two  measures  with  actual  data,  evaluated  the  correlation  between  aircraft  availa¬ 
bility  and  percent  defects  cind  cycle  time,  and  analyzed  management  perceptions 
addressing  the  usefulness  of  these  measures  for  field  use.  Performance  of  this 
analvsis  resulted  in  support  for  these  measures  by  over  two  thirds  of  the  critical 
item  managers  interviewed.  As  a  result,  this  study  concluded  that  both  of  these 
measures,  percent  defects  and  cycle  time,  could  be  useful  to  the  Air  Force  in 
managina  the  CIP  effectively. 


14.  SUBJECT  TERMS 

^  Criticality,  Duality  Control,  Performance  Measures,  Critical 

Item,  Continuous  Improvement 

15.  NUMBER  OF  PAGES 

175  ! 

16.  PRICE  CODE 

17.  SECURITY  CLASSIFICATION 

18  SECURITY  CLASSIFICATION 

19.  SECURITY  CLASSIFICATION 

20.  LIMITATION  OF  ABSTRACT 

OF  REPORT 

OF  THIS  PAGE 

OF  ABSTRACT 

Unclassified 

Unclassified 

Unclassified 

1 _ 

UL 

MSN  7540-0‘ -280-5500  Staroa^d  298  Rev  2 


AFIT  Control  Number  AFIT/GLM/ISC/91  3-40 


AFIT  RESEARCH  ASSESSMENT 


Tbe  purpose  of  this  questionnaire  Is  to  determine  the  potential  for  cur¬ 
rent  and  future  applications  of  AFIT  thesis  research.  Please  return 
completed  questionnaires  to:  AFIT/LSC.  Uright-Patterson  AFB  OH 
45433-6583. 

1.  Old  this  research  contribute  to  a  current  research  project? 
a.  Yes  b.  No 


2.  Oo  you  believe  this  research  topic  is  significant  enough  that  It  would 
have  been  researched  (or  contracted)  by  your  organization  or  another 
agency  If  AFIT  had  not  researched  It? 

a.  Yes  b.  No 


3.  The  benefits  of  AFIT  research  can  often  be  expressed  by  the  equivalent 
value  that  your  agency  received  by  virtue  of  AFIT  performing  the  research. 
Please  estimate  what  this  research  would  have  cost  In  terms  of  manpower 
and/or  dollars  If  It  had  been  accomplished  under  contract  or  If  It  had 
been  done  In-house. 

Man  Years  ^ _  $ _ 


4.  Often  It  Is  not  possible  to  attach  equivalent  dollar  values  to 
research,  although  the  results  of  the  research  may.  In  fact,  be  important. 
Whether  or  not  you  were  able  to  establish  an  equivalent  value  for  this 
research  (3  above),  what  is  your  estimate  of  Its  significance? 

a.  Highly  b.  Significant  c.  Slightly  d.  Of  No 

Significant  Significant  Significance 


5.  Comments 


Name  and  tirade 


IT 


FosTt 


rganization 


